Face recognition using quantized machine learning models

Abstract

An image processing method, comprising acquiring an image captured by an image sensor; detecting, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning mode; cropping the captured image, wherein the cropped image includes the face region; inputting the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model; extracting, by the second trained machine learning model, a first feature of the face region in the cropped image; comparing the first feature with a second feature of a face stored in a memory; determining whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold; and in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model.

Description

FACE RECOGNITION USING QUANTIZED MACHINE LEARNING MODELS

[0001] The present technology relates to a technology for an inference device, an inference method, a program, and an inference system that perform inference using a plurality of machine learning models.

[0002] Edge computing has been known as a technology to reduce processing load on servers.　In the edge computing, various innovations have been observed to make use of computer resources that are less abundant than server devices. For example, when inference processing using DNN (Deep Neural Network) is performed as edge computing, a machine learning model downsized by quantization is used to perform the inference processing.　Thus, a processing speed can be increased, while realizability is secured in exchange for of inference accuracy (see, for example, PTL 1 below).

[0003] WO 2022 / 064656Summary

[0004] Meanwhile, a plurality of machine learning models are used in some cases.　An example refers to a case where an inference result of a machine learning model in a preceding stage is input to a machine learning model in a subsequent stage to perform further inference.

[0005] In such a case where a plurality of stages of machine learning models are used, accuracy in an estimation result of a machine learning model in a preceding stage has a large influence on accuracy in an estimation result of a machine learning model in a subsequent stage.　Therefore, the machine learning models are requested to be suitably quantized.

[0006] The present technology has been made in view of such a problem and has an object of increasing inference accuracy when a plurality of machine learning models are used.Solutions to Problem

[0007] A non-transitory computer-readable storage medium storing computer-readable instructions thereon which, when executed by an information processing device, cause the information processing device to perform a method, the method comprising acquiring an image captured by an image sensor; detecting, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning model; cropping the captured image, wherein the cropped image includes the face region; inputting the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model; extracting, by the second trained machine learning model, a first feature of the face region in the cropped image; comparing the first feature with a second feature of a face stored in a memory; determining whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold; and　　　in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model.

[0008] An image processing method, comprising acquiring an image captured by an image sensor; detecting, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning mode; cropping the captured image, wherein the cropped image includes the face region; inputting the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model; extracting, by the second trained machine learning model, a first feature of the face region in the cropped image; comparing the first feature with a second feature of a face stored in a memory; determining whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold; and in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model.

[0009] The image processing method further comprising capturing a plurality of images with an exposure setting changed for each of the plurality of captured images.

[0010] The image processing method further comprising sequentially capturing an intermediate image having an exposure setting representing an average brightness value within a field of a view of a camera comprising the image sensor, an underexposed image having an exposure setting with which underexposure is greater than the intermediate image, and an overexposed image having an exposure setting with which overexposure is greater than the intermediate image.

[0011] An imaging processing system, comprising an image sensor configured to capture an image; and processing circuitry configured to acquire an image captured by an image sensor, detect, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning model, crop the captured image, wherein the cropped image includes the face region, input the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model, extract, by the second trained machine learning model, a first feature of the face region in the cropped image, compare the first feature with a second feature of a face stored in a memory, determine whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold, and in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model.

[0012] An inference device according to the present technology includes: a first inference processing unit that performs inference by a first machine learning model; and a second inference processing unit that performs inference by a second machine learning model using an inference result of the first inference processing unit, wherein a quantization bit count of a parameter used in the first machine learning model is set to be larger than a quantization bit count of a parameter used in the second machine learning model. According to the present configuration, a quantization bit count of a parameter of a first machine learning model in a preceding stage is set to be large regardless of a second machine learning model.

[0013] An inference method according to the present technology performed by a computer device includes: first inference processing to perform inference by a first machine learning model having a parameter quantization bit count of N; and second inference processing to perform inference by a second machine learning model having a parameter quantization bit count of M smaller than N and using an inference result in the first inference processing.

[0014] A program according to the present technology causes an information processing device to perform: first inference processing to perform inference by a first machine learning model having a parameter quantization bit count of N; and second inference processing to perform inference by a second machine learning model having a parameter quantization bit count of M smaller than N and using an inference result in the first inference processing.

[0015] An inference system according to the present technology includes: a first device including a first inference processing unit that performs inference by a first machine learning model having a parameter quantization bit count of N; and a second device including a second inference processing unit that performs inference by a second machine learning model having a parameter quantization bit count of M smaller than N and using an inference result by the first inference processing unit.

[0016] The functions described above can also be obtained by such an inference method, a program, or an inference system.

[0017] Fig. 1 is a block diagram showing a configuration example of a camera device.Fig. 2 is a diagram showing an example of a frame image and a crop image.Fig. 3 is a diagram for describing deployment of lightweight AI models to the camera device.Fig. 4 is a flowchart showing an example of processing performed by a server device in a first embodiment.Fig. 5 is a flowchart showing an example of processing performed by the camera device in the first embodiment.Fig. 6 is an explanatory diagram of a normal mode and a bracket mode.Fig. 7 is an explanatory diagram showing an example of an intermediate image, an underexposed image, and an overexposed image.Fig. 8 is an explanatory diagram about a blur.Fig. 9 is a flowchart showing an example of processing performed by a camera device in a second embodiment.Fig. 10 is a flowchart showing an example of processing performed by the camera device in a modified example of the second embodiment.Fig. 11 is a diagram showing a relationship between different exposure settings and captured images.Fig. 12 is a flowchart showing an example of processing performed by a camera device in a third embodiment.Fig. 13 is a flowchart showing an example of processing performed by the camera device in a modified example of the third embodiment.Fig. 14 is a diagram showing a configuration example of an inference system.

[0018] Hereinafter, embodiments will be described in the following order. <1. Configuration of Inference Device> <2. Quantization of AI Models> <3. Processing Flow> <3-1. Server Device> <3-2. Camera Device> <4. Second Embodiment> <5. Third Embodiment> <6. Modified Examples> <7. Summary> <8. Present Technology>

[0019] <1. Configuration of Inference Device> In a first embodiment, a camera device 1A will be provided as an example of an inference device 1. A configuration example of the camera device 1A is shown in Fig. 1.

[0020] As shown in Fig. 1, the camera device 1A includes an imaging optical system 31, an optical system driving unit 32, an image sensor IS, a sensor external control unit 33, a memory unit 34, and a communication unit 35.　The image sensor IS and the sensor external control unit 33, the memory unit 34, and the communication unit 35 are connected to each other via a bus 36, and capable of performing data communication each other.

[0021] The imaging optical system 31 includes a lens such as a cover lens, a zoom lens, and a focus lens or an aperture (iris) mechanism.　By the imaging optical system 31, light (incident light) from a subject is guided and condensed on a light-receiving surface of the image sensor IS.

[0022] The optical system driving unit 32 is one comprehensively shown as a driving unit for the zoom lens, the focus lens, and the aperture mechanism of the imaging optical system 31.　Specifically, the optical system driving unit 32 has an actuator for driving each of the zoom lens, the focus lens, and the aperture mechanism and a driving circuit for the actuator.

[0023] The sensor external control unit 33 is configured to include a microcomputer having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) and performs entire control of the camera device 1A by performing various processing according to a program stored in the ROM or a program loaded into the RAM with the CPU.

[0024] Further, the sensor external control unit 33 provides driving instructions for the zoom lens, the focus lens, the aperture mechanism, or the like to the optical system driving unit 32.　The optical system driving unit 32 performs movement of the focus lens or the zoom lens, opening / closing of an aperture blade of the aperture mechanism, or the like according to the driving instructions.

[0025] Further, the sensor external control unit 33 performs control about writing and reading of various data with respect to the memory unit 34. The memory unit 34 is a non-volatile storage device such as, for example, an HDD (Hard Disk Drive) and a flash memory and used as a storage destination (recording destination) of image data output from the image sensor IS.

[0026] In addition, the sensor external control unit 33 performs data communication with an external device via the communication unit 35.　In this example, the communication unit 35 may be capable of deploying, for example, an AI (Artificial Intelligence) model that will be described later via communication with a server device.

[0027] The image sensor IS is configured as, for example, a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor, or the like, and capable of capturing, for example, an RGB image.　Note that the image sensor IS may also be an IR sensor having sensitivity in an IR (Infrared) light band.

[0028] The image sensor IS includes an imaging unit 41, an image signal processing unit 42, a sensor internal control unit 43, a memory unit 44, and a communication I / F 45, each of which is capable of performing data communication each other via a bus 46.

[0029] The imaging unit 41 includes a pixel array unit where pixels having a photoelectric conversion element such as a photodiode are two-dimensionally arrayed and a read-out circuit that reads out an electric signal obtained by photoelectric conversion from each of the pixels of the pixel array unit, and is capable of outputting the electric signal as a captured image signal.

[0030] The read-out circuit performs, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, or the like, and further performs A / D (Analog / Digital) conversion processing for an electric signal obtained by photoelectric conversion.

[0031] The image signal processing unit 42 performs pre-processing, synchronization processing, YC generation processing, resolution conversion processing, codec processing, or the like for a captured image signal as digital data after A / D conversion processing.　In the pre-processing, clamp processing to clamp a black level of R, G, B at a prescribed level with respect to a captured image signal, correction processing between color channels of RGB, or the like is performed.　In the synchronization processing, color separation processing to make image data for respective pixels have all color components of R, G, B is performed.　For example, in the case of imaging elements using color filters in Bayer array, demosaic processing is performed as the color separation processing.　In the YC generation processing, a brightness (Y) signal and a color (C) signal are generated (separated) from R, G, B image data.　In the resolution conversion processing, resolution conversion processing is performed for image data that has undergone various signal processing. In the codec processing, coding processing for recording or communication and file generation are, for example, performed for image data that has undergone the above various processing.　The codec processing enables file generation in, for example, a format such as MPEG-2 (MPEG: Moving Picture Experts Group) and H.264.　Further, generation of a file in a format such as JPEG (Joint Photographic Experts Group), TIFF (Tagged Image File Format), and GIF (Graphics Interchange Format) as a still-image file is also assumed.　Note that in a case where the image sensor IS is a ranging sensor, the image signal processing unit 42 calculates, for example, distance information on a subject on the basis of two signals output from the image sensor IS as iToF (indirect Time of Flight) and outputs a distance image.

[0032] Note that image data output from the image signal processing unit 42 will be occasionally described as simply a “captured image” or a “frame image Gfr” in the following description.

[0033] The sensor internal control unit 43 includes an imaging control unit 51 and an AI image processing unit 52.

[0034] The imaging control unit 51 provides instructions to the imaging unit 41 to perform execution control of an imaging operation.　Similarly, the imaging control unit 51 performs execution control of processing for the image signal processing unit 42 as well.

[0035] The AI image processing unit 52 performs image processing by performing inference processing using an AI model (machine learning model) for a captured image.　In the following description, image processing using AI will be described as “AI image processing.”　The AI image processing is, for example, detection processing to detect a specific subject captured in an image.

[0036] The AI image processing unit 52 detects a region where an image of a specific subject detected by AI image processing is captured, for example, a face region ARf where an image of a person’s face is captured, and outputs coordinate information on the region.

[0037] The AI image processing unit 52 cuts out a face region ARf as a crop image Gcr on the basis of coordinate information indicating a detected face region ARf (see Fig. 2).

[0038] The AI image processing unit 52 is achieved by, for example, a DSP (Digital Signal Processor).

[0039] The memory unit 44 is usable as a so-called frame memory where captured image data (RAW image data) obtained by the image signal processing unit 42 or image data after synchronization processing is stored.　Further, the memory unit 44 is also usable to temporarily store data that is used by the AI image processing unit 52 in the process of AI image processing.

[0040] Further, in the memory unit 44, information on an AI application or an AI model that is used in the AI image processing unit 52 is stored.　Note that the AI application represents an application using an AI model.　Processing to cut out a crop image Gcr from a frame image Gfr on the basis of an inference result of a face region ARf using an AI model or the like is an example of the AI application.

[0041] The communication I / F 45 is an interface that performs communication with the sensor external control unit 33, the memory unit 34, or the like positioned outside the image sensor IS.　The communication I / F 45 performs communication to acquire a program that is run by the image signal processing unit 42, an AI application or an AI model that is used by the AI image processing unit 52, or the like from an external device such as a server device, and causes the same to be stored in the memory unit 44 of the image sensor IS. Thus, an AI model is stored in a part of the memory unit 44 of the image sensor IS, and usage of the AI model by the AI image processing unit 52 is made possible.

[0042] By performing prescribed AI image processing using an AI application or an AI model thus obtained, the AI image processing unit 52 performs detection or the like of a subject according to an intended purpose.

[0043] Information on a crop image Gcr obtained by AI image processing is output to the outside of the image sensor IS via the communication I / F 45.

[0044] That is, it is possible to output image data output from the image signal processing unit 42 via the communication I / F 45 of the image sensor IS, but is also possible to output the image data together with a crop image Gcr generated on the basis of a detection result of AI image processing or output only the crop image Gcr.

[0045] The sensor external control unit 33 performs further AI image processing on a crop image Gcr output from the image sensor IS. Specifically, the sensor external control unit 33 uses an AI model for face authentication.

[0046] Note that an AI model in a preceding stage that is used by the AI image processing unit 52 in the image sensor IS will be defined as a “first AI model M1.”　Further, an AI model in a subsequent stage that is used by the sensor external control unit 33 will be defined as a “second AI model M2.”

[0047] The sensor external control unit 33 performs inference processing using the second AI model M2 and extracts a feature amount of a person’ face captured in a crop image Gcr. Here, the feature amount of the person’s face is, for example, a digitized element different for each person, such as a position, a shape, or depth of a wrinkle, a position or a size of a mole, and a positional relationship for each part.

[0048] The sensor external control unit 33 compares a feature amount of a crop image Gcr extracted by the second AI model M2 with feature amount data of a person stored in the memory unit 34 to perform face authentication.

[0049] For example, if the camera device 1A is a device such as a smart phone possessed by a specific person, a feature amount extracted from a crop image Gcr is compared with a feature amount of a user possessing the smart phone, and a matching rate between the feature amounts is calculated to perform authentication as to whether a person reflected in the crop image Gcr is the possessing user.

[0050] Alternatively, if the camera device 1A is a camera installed in a door of a room that permits access of only prescribed employees, a feature amount of a face extracted from a crop image Gcr is compared with each of feature amounts of the employees permitted to access the room, and unlocking processing is performed when there is any employee having a feature-amount matching rate of a prescribed percentage or more.

[0051] In the memory unit 34 of the camera device 1A, the second AI model M2, an AI application, or the like is stored. The AI application stored in the memory unit 34 includes processing to compare a feature amount extracted as an inference result of the second AI model M2 with a feature amount of a person to calculate a matching rate, or the like.

[0052] Further, the memory unit 34 can be regarded as a database where feature amounts of the faces of persons to be compared are accumulated. Note that the database where the feature amounts are accumulated may be constructed outside the camera device 1A so that the camera device 1A acquires a feature amount to be compared from the database as occasion demands at the time of comparing feature amounts with each other.

[0053] <2. Quantization of AI Models> AI models deployed to the camera device 1A are lightweight AI models obtained by quantizing high-accuracy AI models M0 generated in a server device 2 or the like using abundant computer resources (see Fig. 3).

[0054] When one inference result is obtained using a plurality of AI models as in the present embodiment, it is general to match quantization bit counts of parameters, input data, or the like of the respective AI models. However, if the quantization bit counts of the plurality of AI models are standardized, there is a possibility that inference accuracy reduces. Particularly, if inference accuracy of a first AI model M1 in a preceding stage reduces, a reduction in inference accuracy of a second AI model M2 in a subsequent stage is inevitable.

[0055] Therefore, a quantization bit count of the first AI model M1 is set to be larger than that of the second AI model M2.

[0056] Specifically, the quantization bit count of the first AI model M1 is adjusted to a bit count of image data input to the first AI model M1.

[0057] An example will be provided illustrating a case where the quantization bit count of the second AI model M2 is 8 bits.

[0058] It is assumed that various image data is input to the first AI model M1.　As the image data, a 16-bit RGB image or a 10-bit image in a YUV format is assumed.　The 16-bit RGB image is one where each tone of R, G, B is expressed by 16 bits.

[0059] It is assumed that the quantization bit count of the first AI model M1 that detects a face region ARf using a 16-bit RGB image as an input image is 16 bits.　Specifically, in the server device 2, a high-accuracy AI model M0a for RGB is quantized (quantization bit count = 16 bits) to generate a first AI model M1a for RGB.

[0060] Further, it is assumed that the quantization bit count of the first AI model M1 that detects a face region ARf using a 10-bit image in a YUV format as an input image is 10 bits.　Specifically, in the server device 2, a high-accuracy AI model M0b for YUV is quantized (quantization bit count = 10 bits) to generate a first AI model M1b for YUV.

[0061] Note that a 9-bit image, a 12-bit image, a 16-bit image, or the like other than a 10-bit image exists as an image in a YUV format.　Accordingly, it is desirable to perform quantization after selecting a quantization bit count in accordance with each data format and generate a first AI model M1.

[0062] Further, in a case where image data is output in another data format such as when the image sensor IS is a ToF (Time of Flight) sensor, quantization corresponding to a bit count of the data format is performed with respect to a high-accuracy AI model M0 obtained by performing learning using the output image data in the data format to generate an appropriate first AI model M1.

[0063] The generated first AI models M1 (M1a and M1b) are deployed to the memory unit 44 inside the image sensor IS of the camera device 1A as occasion demands via an arbitrary communication network N (see Fig. 3).

[0064] For example, if the image sensor IS of the camera device 1A outputs a 16-bit RGB image, the first AI model M1a for RGB generated when 16 bits are selected as a quantization bit count is deployed to the memory unit 44 of the image sensor IS.

[0065] For example, if the image sensor IS of the camera device 1A outputs a 10-bit image in a YUV format, the first AI model M1b for YUV that is generated when 10 bits are selected as a quantization bit count is deployed to the memory unit 44 of the image sensor IS.

[0066] The first AI models M1 thus deployed are appropriate AI models corresponding to the data formats of input images, and are models highly well-balanced in both inference accuracy and weight reduction by quantization.

[0067] A crop image Gcr cut out on the basis of a face region ARf as an inference result output from the first AI model M1 is converted into an 8-bit data format so as to be input to the second AI model M2 quantized by 8 bits.

[0068] The converted 8-bit image data is input to the second AI model M2, whereby a feature amount of a face included in the crop image Gcr is extracted.

[0069] Thus, since inference processing in the subsequent-stage second AI model M2 can be performed using a high-accuracy inference result, a reduction in inference accuracy of the second AI model M2 can be suppressed.

[0070] <3. Processing Flow> An example of processing performed by the respective devices in the first embodiment will be described. <3-1. Server Device> An example of processing performed by the server device 2 will be described with reference to Fig. 4. In step S101, a computation processing unit such as a CPU in the server device 2 selects a quantization bit count corresponding to the output format of the image sensor IS.

[0071] In step S102, the computation processing unit of the server device 2 performs quantization of a high-accuracy AI model M0 using the selected quantization bit count to generate a first AI model M1. Further, in step S102, the computation processing unit of the server device 2 generates a second AI model M2 by quantization.

[0072] In step S103, the computation processing unit of the server device 2 deploys the first AI model M1 to the target image sensor IS.

[0073] In step S104, the computation processing unit of the server device 2 deploys the second AI model M2 to the camera device 1A including the target image sensor IS.

[0074] This brings about a state where face detection processing using the first AI model M1 in a preceding stage and face authentication processing using the second AI model M2 in a subsequent stage are executable in the camera device 1A.

[0075] <3-2. Camera Device> Next, an example of processing performed by the camera device 1A is shown in Fig. 5. In step S201, the sensor external control unit 33 of the camera device 1A determines whether a first AI model M1 and a second AI model M2 have been deployed.

[0076] When it is determined that any of the AI models has not been deployed, the sensor external control unit 33 transmits a distribution request for the AI model that has not been deployed to the server device 2. Note that the distribution request in step S202 may be performed by the sensor internal control unit 43 when the AI model that has not been deployed is the first AI model M1.

[0077] After it is determined in step S201 that the first AI model M1 and the second AI model M2 have been deployed or after the processing of step S202 is completed, the imaging control unit 51 of the sensor internal control unit 43 of the camera device 1A performs capturing of a frame image Gfr in step S203.

[0078] In step S204, the AI image processing unit 52 of the sensor internal control unit 43 of the camera device 1A performs first inference processing using the first AI model M1.　Thus, a face region ARf is detected in the frame image Gfr.

[0079] In step S205, the sensor internal control unit 43 of the camera device 1A cuts out a crop image Gcr so as to include the face region ARf.

[0080] In step S206, the sensor external control unit 33 of the camera device 1A performs second inference processing by inputting the crop image Gcr to the second AI model M2.

[0081] In step S207, the sensor external control unit 33 of the camera device 1A performs processing to compare a feature amount of the face region ARf extracted by the second inference processing with a feature amount of a person acquired from the memory unit 34.

[0082] In step S208, the sensor external control unit 33 of the camera device 1A determines whether a matching rate between the feature amounts is a threshold (for example, 80% or the like) or more. When it is determined that the matching rate between the feature amounts is the threshold or more, the sensor external control unit 33 of the camera device 1A performs an authentication success notification in step S209. Note that the sensor external control unit 33 may perform processing to store an authentication success log in the memory unit 34 instead of or in addition to the authentication success notification.

[0083] The authentication success log includes, for example, information such as the feature amount of the face region ARf, the matching rate between the feature amounts, the person to be compared, and the feature amount of the person.　Further, an authentication success date and time may also be included in the authentication success log.

[0084] After the processing of step S209 is completed, the imaging control unit 51 of the sensor internal control unit 43 of the camera device 1A continues to capture a frame image Gfr in step S203.

[0085] When a matching rate of the threshold or more is not obtained for any person to be compared in step S208, the imaging control unit 51 of the sensor internal control unit 43 of the camera device 1A returns to the processing of step S203 to perform capturing of a frame image Gfr.

[0086] Note that when No is determined in step S208, the sensor external control unit 33 of the camera device 1A may return to the processing to step S203 after performing authentication failure notification processing or storage processing to store an authentication failure log.

[0087] <4. Second Embodiment> A second embodiment is aimed principally at improving an authentication probability and authentication accuracy with respect to an AI application that performs face authentication using a processing result of a second AI model M2.

[0088] In the second embodiment, an example of a camera device 1A will be provided as an inference device 1 like the first embodiment. A configuration example of the camera device 1A is described in the first embodiment with reference to Fig. 1, and therefore its duplicated description will be avoided.

[0089] An AI application that is stored in a memory unit 34 of the camera device 1A and performed by a sensor external control unit 33 is used to determine whether a person is one to be recognized, by comparing a feature amount of a face region ARf extracted as an inference result of a second AI model M2 with feature amounts of respective users (employees) stored in the memory unit 34 as a database.

[0090] The AI application will be described as a “face authentication application AP.”

[0091] The face authentication application AP performs face authentication on the basis of one crop image Gcr acquired from one frame image Gfr.　This mode will be defined as a “normal mode Md1.”

[0092] When face authentication AP using one crop image Gcr fails, that is, when there is no user having a feature-amount matching rate of a prescribed percentage or more, transition from the normal mode Md1 to a bracket mode Md2 is enabled (see Fig. 6).

[0093] In the bracket mode Md2, feature amounts extracted from a plurality of crop images Gcr acquired using a plurality of frame images Gfr are added together to determine whether a captured person is one to be recognized.

[0094] Then, when face authentication becomes finally successful in the bracket mode Md2, either the bracket mode Md2 or the normal mode Md1 is resumed after provision of an authentication success notification.

[0095] On the other hand, when the face authentication finally fails in the bracket mode Md2, either the bracket mode Md2 or the normal mode Md1 is resumed after provision of an authentication failure notification.

[0096] Some examples of the transition modes after the respective notification processing are assumed. For example, in a case where it can be determined that a person reflected in a crop image Gcr has been changed, transition to the normal mode Md1 may be enabled.　The case where the person has been changed refers to, for example, a case where the person being tracked disappears out of the field of view and then a person is detected again within the angle of view of the camera device 1A, or the like.

[0097] Further, in a case where the hitting way of light on a subject changes with time, the bracket mode Md2 may be controlled to be resumed until a prescribed time elapses or until a brightness change about a background within the field of view exceeds a threshold, and the normal mode Md1 may be controlled to be resumed when the prescribed time has elapsed or when the brightness change about the background within the angle of view has exceeded the threshold.

[0098] That is, transition to the bracket mode Md2 may be enabled when there is a need to extract feature amounts using a plurality of crop images Gcr to increase the success probability of face authentication processing, and the normal mode Md1 may be enabled when an increase in the success probability of the face authentication processing is expected using only one crop image Gcr as a result of a change in the condition of environment light.

[0099] Alternatively, transition to the normal mode Md1 is necessarily enabled, but transition to the bracket mode Md2 may also be enabled when face authentication using one crop image Gcr does not become successful.

[0100] Here, the bracket mode Md2 will be described. In the bracket mode Md2, a plurality of frame images Gfr are captured with an exposure setting changed.

[0101] Here, the exposure setting refers to integration of settings including a setting with which the brightness of a frame image Gfr can be changed.　That is, the exposure setting includes not only an iris setting or a shutter speed setting but also an ISO sensitivity setting or the like.

[0102] In the bracket mode Md2, an “intermediate image Gn” representing a frame image Gfr captured with an intermediate exposure setting StN representing an exposure setting with which an average brightness value within the field of view of the camera device 1A falls within a certain range, an “underexposed image Gu” representing a frame image Gfr captured with an underexposure setting StU representing a setting with which underexposure occurs to a greater extent than the intermediate exposure setting StN, and an “overexposed image Go” representing a frame image Gfr captured with an overexposure setting StO representing an exposure setting with which overexposure occurs to a greater extent than the intermediate exposure setting StN are sequentially obtained (see Fig. 7).

[0103] In an example shown in Fig. 7, a frame image Gfr (n - 2) representing the second previous frame image Gfr is defined as an underexposed image Gu, a frame image Gfr (n - 1) representing the previous frame image Gfr is defined as an intermediate image Gn, and a frame image Gfr (n) representing the latest frame image Gfr is defined as an overexposed image Go.

[0104] A frame image Gfr (n + 1) representing a frame image Gfr to be next captured may be an underexposed image Gu or an intermediate image Gn again.

[0105] In the respective frame images Gfr, “appropriate exposed regions ARe” representing regions from which feature amounts of a face are suitably extractable and in which exposure is appropriate are different.

[0106] For example, in the underexposed image Gu, a right region (a region surrounded by a dashed line) in a crop image Gcr is defined as an appropriate exposed region ARe.

[0107] Further, in the intermediate image Gn, a region (a region surrounded by a dashed line) at the center in a right-left direction in a crop image Gcr is defined as an appropriate exposed region ARe.

[0108] Furthermore, in the overexposed image Go, a left region (a region surrounded by a dashed line) in a crop image Gcr is defined as an appropriate exposed region ARe.

[0109] In the bracket mode Md2, the face authentication application AP collectively assumes feature amounts extracted from the respective crop images Gcr of the underexposed image Gu, the intermediate image Gn, and the overexposed image Go as an integrated feature amount.

[0110] The face authentication application AP compares an integrated feature amount with a feature amount of a person to be compared acquired from the memory unit 34 and determines the success or failure of face authentication depending on whether a matching rate is a threshold or more.

[0111] Thus, it becomes possible to perform high-accuracy authentication processing even if the condition of environment light of the camera device 1A is poor as an environment for performing face authentication.

[0112] Note that the number of frame images Gfr to be used in calculation of an integrated feature amount is variously considered. For example, all frame images Gfr captured within a prescribed time may be used to calculate an integrated feature amount.

[0113] Alternatively, all frame images Gfr captured during a period in which face regions ARf are continuously detected in first inference processing may be used to calculate an integrated feature amount.

[0114] Furthermore, frame images Gfr selected to include captured images, each containing one image having a different exposure setting may be used to calculate an integrated feature amount.　In the example described above, one latest underexposed image Gu, one latest intermediate image Gn, and one latest overexposed image Go are used to calculate an integrated feature amount.

[0115] Note that frame images Gfr may be selected to include n (such as two or three) captured images each having a different exposure setting to calculate an integrated feature amount.

[0116] The face authentication application AP may determine whether an index value for each crop image Gcr is used to be applied to second inference processing by the second AI model M2, specifically, to calculation of an integrated feature amount.

[0117] Some examples of index values used for determining whether to use in inference processing will be provided.

[0118] In a first example, likelihood information output as a result of inference processing by the first AI model M1 is used as an index value.

[0119] For example, when the likelihood information is lower than a threshold, generation of a crop image Gcr from a frame image Gfr concerned may be avoided, or it may be determined not to perform second inference processing for a generated crop image Gcr and not to perform calculation of a feature amount.

[0120] In a second example, an evaluation value of a blur is used as an index value.

[0121] For example, as shown in Fig. 8, an index value decreases when a large blur occurs in a crop image Gcr.　As a result, second inference processing is hardly performed.　Further, when a small blur occurs or no blur occurs in a crop image Gcr, an index value increases.　As a result, second inference processing is performed, and a feature amount is easily calculated.

[0122] Besides, the face authentication application AP may calculate an index value using a noise amount in a crop image Gcr and determine whether calculation of a feature amount is performed by second inference processing.

[0123] Note that as for a crop image Gcr determined not to be used in the second AI model M2, second inference processing may be performed using an alternate crop image Gcr to calculate a feature amount to calculate an integrated feature amount.

[0124] For example, when a large blur occurs in a crop image Gcr of an overexposed image Go, an integrated feature amount may be calculated using a crop image Gcr of an overexposed image Go that has been previously captured as an alternate crop image Gcr.

[0125] Alternatively, an overexposed image Go may be acquired with application of an exposure setting for performing capturing of an alternate image to the next imaging to generate a crop image Gcr.

[0126] An example of processing performed by the camera device 1A in the second embodiment is shown in Fig. 9. Note that the example of the processing shown in Fig. 9 refers to an example of a case where respective AI models have been deployed to the camera device 1A.　When the AI models have not been deployed, the processing of step S202 in Fig. 5 is performed to first deploy the AI models.

[0127] Descriptions of the processing described above with reference to Fig. 5 among the respective processing shown in Fig. 9 will be omitted to avoid duplicated descriptions.

[0128] In step S221, the sensor external control unit 33 of the camera device 1A performs an exposure setting.　The exposure setting performed here varies depending on whether a frame image Gfr to be captured corresponds to an intermediate image Gn, an overexposed image Go, or an underexposed image Gu.

[0129] Next, in step S203, an imaging control unit 51 of the camera device 1A performs capturing of a frame image Gfr.

[0130] In step S204, an AI image processing unit 52 of a sensor internal control unit 43 of the camera device 1A performs first inference processing using a first AI model M1.

[0131] In step S205, the sensor internal control unit 43 of the camera device 1A cuts out a crop image Gcr from the frame image Gfr.

[0132] In step S222, the sensor external control unit 33 of the camera device 1A calculates or acquires an index value of the crop image Gcr. The index value calculated or acquired here may be, for example, an index value indicating the strength of a blur, an index value indicating the likelihood of an inference result of the first inference processing, or any other index value (such as an index value indicating a noise amount).

[0133] In step S223, the sensor external control unit 33 of the camera device 1A determines whether the index value is a threshold or more.

[0134] When it is determined that the index value is the threshold or more, the sensor external control unit 33 of the camera device 1A performs second inference processing in step S206. On the other hand, when it is determined that the index value is less than the threshold, the sensor external control unit 33 of the camera device 1A avoids the processing of step S206.

[0135] In step S224, the sensor external control unit 33 of the camera device 1A adds feature amounts extracted from a plurality of crop images Gcr together to calculate an integrated feature amount.

[0136] In step S207, the sensor external control unit 33 of the camera device 1A compares the integrated feature amount with a feature amount of a person acquired from the memory unit 34.

[0137] In step S208, the sensor external control unit 33 of the camera device 1A determines whether a matching rate between the feature amounts is a threshold or more.　When it is determined that the matching rate is the threshold or more, the sensor external control unit 33 performs an authentication success notification in step S209 and returns to step S221.

[0138] On the other hand, when it is determined that the matching rate between the feature amounts is less than the threshold, the sensor external control unit 33 returns to step S221 without performing the processing of step S209.

[0139] Here, a processing example of a case where an alternate frame image Gfr is captured is shown in Fig. 10 as a modified example.

[0140] The respective units of the camera device 1A perform the respective processing of steps S221, S203, S204, S205, and S222.　The sensor external control unit 33 of the camera device 1A determines whether an index value is a threshold or more in step S223.

[0141] Processing performed by the camera device 1A when it is determined that the index value is the threshold or more is the same as that of Fig. 9, and therefore its description will be omitted.

[0142] When it is determined that the index value is less than the threshold, the sensor external control unit 33 of the camera device 1A determines in step S241 whether to perform capturing of an alternate frame image Gfr.

[0143] When the capturing of the alternate frame image Gfr is performed, the sensor external control unit 33 of the camera device 1A returns to step S221 to perform an exposure setting for capturing the acquired frame image Gfr and performs capturing of the frame image Gfr in step S203.

[0144] On the other hand, when it is determined that the capturing of the alternate frame image Gfr is not performed, for example, when it is determined that a feature amount obtained on the basis of a frame image Gfr captured before the present time is used, the sensor external control unit 33 of the camera device 1A adds feature amounts extracted from a plurality of crop images Gcr together to calculate an integrated feature amount in step S224.

[0145] <5. Third Embodiment> In a third embodiment, the order of capturing frame images Gfr having a different exposure setting is considered.

[0146] In the present embodiment, an intermediate image Gn is acquired using an intermediate exposure setting StN, and nine types of underexposed images Gu1, Gu2, …, Gu8, and Gu9 having different exposure are acquired using nine types of underexposure settings StU1, StU2, …, StU8, and StU9. Furthermore, nine types of overexposed images Go1, Go2, …, Go8, and Go9 having different exposure are acquired using nine types of overexposure settings StO1, StO2, … StO8, and StO9.

[0147] Note that the respective images are arranged in ascending order of exposure, with the underexposed images shown as Gu9, Gu8, … Gu2, and Gu1, the intermediate image shown as Gn, and the overexposed images shown as Go1, Go2, …, Go8, and Go9 (see Fig. 11).

[0148] First, the intermediate image Gn is captured as a frame image Gfr before a person as a subject is present within the field of view and a crop image Gcr based on a result of first inference processing is cut out.

[0149] Next, when a person as a subject is reflected within the field of view and the process of cutting out a crop image Gcr starts, an exposure setting is changed to obtain any of the underexposed image Gu9 with the lowest exposure level and the overexposed image Go9 with the highest exposure level in the next frame image Gfr.

[0150] The selection of the frame image Gfr from among the underexposed image Gu9 and the overexposed image Go9 may be performed on the basis of, for example, an average brightness value of the crop image Gcr.

[0151] For example, when the average brightness value is high, there may be many pixels that are too bright, which results in the failure of extracting feature amounts of bright pixel regions.　Accordingly, the underexposed image Gu9 is selected to suitably extract the feature amounts of the bright pixel regions. On the other hand, when the average brightness value is low, there may be many pixels that are too dark, which results in the failure of extracting feature amounts of dark pixel regions.　Accordingly, the overexposed image Go9 is selected to suitably extract the feature amounts of the dark pixel regions.

[0152] When the underexposed image Gu9 is selected, in the next frame image Gfr, the underexposed image Gu4 or the underexposed image Gu5 may be, for example, selected as intermediate exposure between the intermediate image Gn and the underexposed image Gu9, or the underexposed image Gu8 may be selected so as to gradually decrease exposure.

[0153] Further, the overexposed image Go9 may be selected after the underexposed image Gu9, so that the frame images Gfr of both ends (the underexposed image Gu9 and the overexposed image Go9) in the exposure setting are obtained in sequence.

[0154] An example of processing performed by the camera device 1A to perform an exposure setting for each frame image Gfr is shown in Fig. 12.　Note that processing performed by the camera device 1A to perform first inference processing and second inference processing using a frame image Gfr captured according to the exposure setting is the same as that shown in Fig. 5 and therefore its description will be omitted.

[0155] In step S301, the sensor external control unit 33 of the camera device 1A first selects an intermediate exposure setting StN.　Thus, an intermediate image Gn is obtained as a frame image Gfr. Note that an exposure setting in step S301 may be selected so that an average brightness value of the frame image Gfr falls within a certain range. For example, an exposure setting for obtaining the intermediate image Gn during the day may be adjusted to reduce exposure than an exposure setting for obtaining the intermediate image Gn during the night.

[0156] In step S302, the sensor external control unit 33 of the camera device 1A determines whether a face region ARf has been detected.

[0157] When it is determined that the face region ARf has not been detected, the sensor external control unit 33 of the camera device 1A repeatedly performs the processing of step S302. Thus, the process of capturing a frame image Gfr is repeatedly performed with the intermediate exposure setting StN remaining unchanged until the face region ARf has been detected.

[0158] When it is determined that the face region ARf has been detected, the sensor external control unit 33 of the camera device 1A calculates an average brightness value of a crop image Gcr in step S303.

[0159] Next, in step S304, the sensor external control unit 33 of the camera device 1A determines whether the average brightness value is a threshold or more.

[0160] When it is determined that the average brightness value is the threshold or more, the sensor external control unit 33 of the camera device 1A determines that feature amounts of bright pixel regions are hardly extractable and therefore selects an underexposure setting StU in step S305.　The underexposure setting StU selected here is, for example, an underexposure setting StU9 with the lowest exposure level.

[0161] On the other hand, when it is determined that the average brightness value is less than the threshold, the sensor external control unit 33 of the camera device 1A determines that feature amounts of dark pixel regions are hardly extractable and therefore selects an overexposure setting StO in step S306.　The overexposure setting StO selected here is, for example, an overexposure setting StO9 with the highest exposure level.

[0162] After performing the processing of any of step S305 and step S306, the sensor external control unit 33 of the camera device 1A determines whether face authentication processing by a face authentication application AP has become successful in step S307.

[0163] When it is determined that the face authentication processing has become successful, the sensor external control unit 33 of the camera device 1A returns to step S301.　After returning to the intermediate exposure setting StN, the sensor external control unit 33 repeatedly performs a series of the processing shown in Fig. 12.

[0164] When it is determined that the face authentication processing has failed, the sensor external control unit 33 of the camera device 1A explores all prepared exposure settings to determine whether characteristic points have been extracted using all images (underexposed images Gu, an intermediate image Gn, and overexposed images Go) in step S308.

[0165] When it is determined that all the exposure settings have been explored, the sensor external control unit 33 of the camera device 1A returns to the processing of step S301.

[0166] On the other hand, when it is determined that all the exposure settings have not been explored, the sensor external control unit 33 of the camera device 1A switches to an unset exposure setting in step S309.　After capturing a frame image Gfr, the sensor external control unit 33 returns to the determination processing of step S307.

[0167] Note that the unset exposure setting is changed to a set exposure setting when selected as a new exposure setting in any of steps, S305, S306, and S308.　Then, the set exposure setting is restored to the unset exposure setting at the time of returning to the processing of step S301.

[0168] A modified example of the third embodiment will be described with reference to Fig. 13.

[0169] Note that the same processing as that of Fig. 12 will be denoted by the same step numbers and its description will be appropriately omitted.

[0170] After selecting an intermediate exposure setting StN in step S301, the sensor external control unit 33 of the camera device 1A determines whether a face region ARf has been detected in step S302.

[0171] When it is determined in step S302 that the face region ARf has not been detected, the sensor external control unit 33 of the camera device 1A proceeds to step S321 and selects an exposure setting to increase exposure by one level.　Thus, the sensor external control unit 33 selects an overexposure setting StO1 after selecting the intermediate exposure setting StN.　When performing the processing of step S321 again, the sensor external control unit 33 selects an overexposure setting StO2.

[0172] Note that the sensor external control unit 33 selects an underexposure setting StU9 after selecting an overexposure setting StO9.　That is, the sensor external control unit 33 sequentially selects one of the exposure settings shown in Fig. 11, while shifting the same rightwards.　After selecting the exposure setting at the right end, the sensor external control unit 33 selects the exposure setting at the left end.　In this manner, the sensor external control unit 33 repeatedly performs a series of the exposure settings for each frame image Gfr. Note that the sensor external control unit 33 may perform the selection of the exposure settings, while shifting each of the settings shown in Fig. 11 leftwards.

[0173] When it is determined in step S302 that the face region ARf has been detected, the sensor external control unit 33 of the camera device 1A determines whether a current exposure setting is an overexposure setting StO or an underexposure setting StU in step S322.

[0174] When it is determined that the current exposure setting is the overexposure setting StO, the sensor external control unit 33 of the camera device 1A selects the underexposure setting StU in step S305.

[0175] On the other hand, when it is determined that the current exposure setting is the underexposure setting StU, the sensor external control unit 33 of the camera device 1A selects the overexposure setting StO in step S306.

[0176] <6. Modified Examples> The above example provides an example of sharing processing so that the sensor internal control unit 43 performs face detection by first inference processing and cut-out processing of a crop image Gcr and the sensor external control unit 33 performs feature amount extraction by second inference processing and face authentication processing by a face authentication application AP.

[0177] There are also methods for sharing performing other than the above method. For example, the sensor internal control unit 43 performs face detection by first inference processing and cut-out processing of a crop image Gcr, and the sensor external control unit 33 performs feature amount extraction by second inference processing. Data of extracted feature amounts may be transmitted to the server device 2 so that the server device 2 performs face authentication processing by a face authentication application AP. The server device 2 may be a cloud server device or an on-premises server device.

[0178] Further, the above example provides an example of making transition between a normal mode Md1 and a bracket mode Md2, but an operation may be performed in the bracket mode Md2 at any time.

[0179] That is, feature amounts extracted from a plurality of frame images Gfr having a different exposure setting may be added together at any time to perform face authentication processing.

[0180] Further, as shown in Fig. 14, first inference processing and cut-out processing of a crop image Gcr may be performed by the camera device 1A (for example, the sensor internal control unit 43), and second inference processing and face authentication processing may be performed by an information processing device 3 such as a personal computer connected to the camera device 1A, or the like. Further, the information processing device 3 may correspond to the server device 2. That is, an inference system may include a configuration as shown in Fig. 14.

[0181] The above example assumes first inference processing using a first AI model M1 as face detection processing and second inference processing using a second AI model M2 as processing to extract feature amounts of a face.　However, the first inference processing and the second inference processing may include other processing. For example, the first inference processing may include processing to determine whether person’s voice is contained in sound data, and the second inference processing may include processing to extract feature amounts of the person’s voice contained in the sound data.　In this case, a voiceprint authentication application AP’ may be used as an AI application to identify an individual on the basis of the person’s voice contained in the sound data.

[0182] <7. Summary> The inference device 1 (the camera device 1A) according to the present technology includes: a first inference processing unit (for example, the AI image processing unit 52) that performs inference by a first machine learning model (the first AI model M1, the first AI model M1a for RGB, and the first AI model M1b for YUV); and a second inference processing unit (for example, the sensor external control unit 33) that performs inference by a second machine learning model (the second AI model M2) using an inference result of the first inference processing unit. Further, a quantization bit count of a parameter used in the first machine learning model is set to be larger than that used in the second machine learning model. For example, when inference processing is continuously performed using two machine learning models, quantization is performed in accordance with a small machine learning model.　This brings about a problem that inference accuracy cannot be increased. According to the present configuration, a quantization bit count of a parameter of the first machine learning model in a preceding stage is set to be large regardless of the second machine learning model. Thus, accuracy in an inference result of the first machine learning model in the preceding stage can be increased.　Further, when the inference result of the first machine learning model is input to the second machine learning model, inference accuracy of the second machine learning model can also be increased as a result of the increase in the inference accuracy of the first machine learning model.

[0183] As described with reference to the respective figures of Figs. 1 to 3 or the like, a first machine learning model (the first AI model M1, the first AI model M1a for RGB, and the first AI model M1b for YUV) may be used in processing to infer a region (for example, a face region ARf) containing an object (for example, a person’s face) in an input image (for example, a frame image Gfr), and a second machine learning model (the second AI model M2) may be used in processing to recognize the object (the person’s face) contained in the region in the inference device 1 (the camera device 1A). Thus, when object detection and object recognition for an image are performed using machine learning models, accuracy can be increased in both inference processing in a preceding stage and inference processing in a subsequent stage.

[0184] As described with reference to the respective figures of Figs. 1 to 3 or the like, a first machine learning model (the first AI model M1, the first AI model M1a for RGB, and the first AI model M1b for YUV) may be used in processing to infer a face region ARf containing a person’s face in an input image (for example, a frame image Gfr), and a second machine learning model (the second AI model M2) may be used in verification between the face contained in the face region ARf and data contained in a database (for example, the memory unit 34) in the inference device 1 (the camera device 1A). That is, the first machine learning model is defined as a model for face detection, and the second machine learning model is defined as a model for face authentication. Thus, estimation of a face region ARf with high accuracy is made possible, and the success rate of face authentication processing can be increased. Further, a possibility that a region not actually containing a face is falsely determined as a face region ARf is reduced, whereby unnecessary inference processing can be avoided, and power consumption of the inference device 1 can be reduced.　Particularly, it is preferable for the inference device 1 to achieve its long-time drive by the effect of reducing power consumption when the inference device 1 is a battery-driven mobile device.

[0185] As described with reference to the respective figures of Figs. 1 to 3 and Fig. 5, or the like, a second machine learning model (the second AI model M2) may be capable of outputting a feature amount of a face contained in a face region ARf, and a second inference processing unit (for example, the sensor external control unit 33) may perform person identification by verifying the output feature amount with feature amount data contained in a database (for example, the memory unit 34) in the inference device 1 (the camera device 1A). Thus, the second machine learning model can be a small model capable of extracting a feature amount. Accordingly, a memory in which the second machine learning model is stored can be downsized.　Particularly, when the inference device 1 is a system including a plurality of devices, and processing before feature amount extraction is performed by a mobile device, while person identification processing in a subsequent stage is performed by another device, it is preferable to achieve miniaturization of the mobile device.

[0186] As described with reference to Figs. 6 and 7 or the like, an input image (for example, a frame image Gfr) may include a plurality of images (an intermediate image Gn, an overexposed image Go, and an underexposed image Gu) having different exposure, and a second inference processing unit (for example, the sensor external control unit 33) may perform person identification using respective feature amounts of the plurality of images in the inference device 1 (the camera device 1A). Thus, it becomes possible to use, as for a feature amount not extracted from one face image, a feature amount extracted from another face image having different exposure. Accordingly, person identification can be suitably performed even in a situation where an optimum exposure setting is difficult.

[0187] As described with reference to Fig. 7 or the like, an input image (for example, a frame image Gfr) may correspond to a frame image Gfr captured by an imaging device (the camera device 1A), a first inference processing unit (for example, the AI image processing unit 52) may perform processing to infer a face region ARf for each frame image Gfr, and a second inference processing unit (for example, the sensor external control unit 33) may perform person identification using a feature amount output from a second machine learning model (the second AI model M2) about a face region ARf for each of a prescribed number of most recent frame images Gfr in the inference device 1 (the camera device 1A). For example, images are sequentially captured with different exposure settings, and a person captured in a frame image Gfr concerned is identified using a feature amount extracted from the frame image Gfr (the latest frame image Gfr) and feature amounts extracted from a plurality of previously-captured old frame images Gfr having different exposure. Thus, even if exposure of a frame image Gfr concerned is not optimum and an extracted feature amount is small, it becomes possible to perform person identification such as identify verification on the basis of abundant feature amounts extracted from old frame images Gfr. Accordingly, omission of the labor of searching for and adjusting optimum exposure is made possible, and person identification can be efficiently performed. Further, feature amounts are used only from a prescribed number of the most recent frame images Gfr, whereby a possibility that feature amounts of other persons extracted from frame images Gfr that are too old are used can be reduced, and suitable person identification is made possible.

[0188] As described with reference to the respective figures of Figs. 11 to 13 or the like, an input image (for example, a frame image Gfr) may include an intermediate image Gn identified as being intermediately exposed, a plurality of overexposed images Go identified as being further overexposed than the intermediate image Gn, and a plurality of underexposed images Gu identified as being further underexposed than the intermediate image Gn, and a second inference processing unit (for example, the sensor external control unit 33) may perform, after performing person identification about the intermediate image Gn, person identification about any of the most overexposed image (for example, an overexposed image Go9) among the overexposed images Go and the most underexposed image (for example, an underexposed image Gu9) among the underexposed images Gu in the inference device 1 (the camera device 1A). Thus, a possibility that a feature amount is extracted from a next frame image Gfr as for a feature amount not extractable from an intermediate image Gn can be increased. Accordingly, a possibility that the time until successful face authentication is reduced can be increased.

[0189] As described with reference to the respective figures of Figs. 11 to 13 or the like, a second inference processing unit (for example, the sensor external control unit 33) in the inference device 1 (the camera device 1A) may select an image to be used in person identification after an intermediate image Gn on the basis of a brightness value of a region having a small amount of feature extraction in a face region ARf. Thus, for example, when a feature amount of a dark shaded region is not obtained after face authentication is performed on the basis of a feature amount extracted from an intermediate image Gn, a feature amount extracted from an overexposed image Go is added to perform face authentication.　Further, when a feature amount of a bright region irradiated with light is not obtained, a feature amount extracted from an underexposed image Gu is added to perform face authentication. Thus, the time until successful face authentication can be reduced.

[0190] As described with reference to the respective figures of Figs. 8 to 10 or the like, a second inference processing unit (for example, the sensor external control unit 33) may determine whether to input an image of a face region ARf concerned to a second machine learning model (the second AI model M2) according to an index value based on the face region ARf supplied from a first inference processing unit (for example, the AI image processing unit 52) in the inference device 1 (the camera device 1A). Thus, an image that is unnecessary or obstructive for inference processing is eliminated, and the success rate of face authentication processing can be increased.

[0191] As described with reference to Fig. 8 or the like, an index value may be defined as an evaluation value about a blur occurring in a face region ARf, and a second inference processing unit (for example, the sensor external control unit 33) may determine that an image of the face region ARf concerned is not input to a second machine learning model (the second AI model M2) when the index value indicates the occurrence of a blur of a prescribed size or more in the inference device 1 (the camera device 1A). Thus, the success rate of face authentication processing can be increased.

[0192] As described above, an index value may be defined as an evaluation value indicating the likelihood of an inference result of a first inference processing unit (for example, the AI image processing unit 52), and a second inference processing unit (for example, the sensor external control unit 33) may determine that an image of a face region ARf concerned is not input to a second machine learning model (the second AI model M2) when the index value is less than a threshold in the inference device 1 (the camera device 1A). Thus, a low-accuracy face region ARf detected as an inference result of face detection processing does not become an input image (for example, a frame image Gfr) of the second inference processing unit and is eliminated.　Therefore, the success rate of face authentication processing can be increased.

[0193] As described with reference to the respective figures of Figs. 1 to 3 or the like, the inference device 1 may be defined as the camera device 1A including the image sensor IS having the sensor internal control unit 43 as a first inference processing unit (for example, the AI image processing unit 52) and the sensor external control unit 33 that is provided as a second inference processing unit (for example, the sensor external control unit 33) and controls an imaging operation by the image sensor IS, and an image input to a first machine learning model (the first AI model M1, the first AI model M1a for RGB, and the first AI model M1b for YUV) may be defined as a captured image captured by the image sensor IS. A region where a specific subject is captured in an image captured by the image sensor IS is detected. Accordingly, it is unnecessary to input an input image (for example, a frame image Gfr) from the outside of the image sensor IS, which enables an increase in the processing speed of inference processing.

[0194] As described with reference to the respective figures of Figs. 1 to 3 or the like, the sensor internal control unit 43 of the camera device 1A serving as the inference device 1 may perform processing to infer a face region ARf containing a person’s face in a captured image by inputting the captured image to a first machine learning model (the first AI model M1, the first AI model M1a for RGB, and the first AI model M1b for YUV), and the sensor external control unit 33 may extract a feature amount of a face contained in the face region ARf by inputting an image of the face region ARf to a second machine learning model (the second AI model M2) and perform person identification by verifying the feature amount with feature amount data contained in a database (for example, the memory unit 34). Thus, a face region ARf is detected inside the image sensor IS, and person identification outside the image sensor IS and inside the camera device or the like is made possible. Thus, a face image or the like is not output to the outside of the camera device 1A, and privacy protection of a subject is made possible.

[0195] As described with reference to Fig. 6 or the like, the sensor external control unit 33 in the inference device 1 (the camera device 1A) may transition to a bracket mode Md2 to capture a plurality of images having different exposure when verification has failed, and perform person identification using feature amounts of faces contained in face regions ARf in the plurality of images having the different exposure by a second machine learning model (the second AI model M2). Thus, face detection processing and face authentication processing are performed on a basis of an image captured with a normal exposure setting (an intermediate exposure setting StN).　Then, when the face authentication has become successful, transition to the bracket mode Md2 is not performed. That is, an unnecessary mode change is not performed when the condition of environment light is suitable and transition to the bracket mode Md2 is unnecessary, and processing load on the inference device 1 can be reduced. Further, when the condition of environment light is not suitable and face authentication does not become successful with a normal exposure setting, the success rate of the face authentication can be increased by transitioning to the bracket mode Md2. Further, the labor of adjusting the exposure setting by a user is unnecessary in this case, and therefore quick face authentication is enabled.

[0196] As described with reference to Fig. 3 or the like, a first machine learning model (the first AI model M1, the first AI model M1a for RGB, and the first AI model M1b for YUV) used by a first inference processing unit of the camera device 1A (for example, the AI image processing unit 52) serving as the inference device 1 may be defined as one using parameters with a quantization bit count corresponding to the data format (such as an RGB format and a YUV format) of a captured image. Thus, for example, when the gradation of respective pixels of a captured image is expressed by 10 bits, a relatively large machine learning model using 10-bit parameters is used to increase inference accuracy. On the other hand, when the gradation of respective pixels of a captured image is expressed by 8 bits, a relatively small machine learning model using 8-bit parameters is used. Thus, a computation amount and a processing time for inference processing can be reduced. That is, it becomes possible to optimize the balance between inference accuracy and a processing time by using machine learning models having parameters corresponding to a bit count of the data format of a captured image. Particularly, an unnecessary reduction in inference accuracy caused when a bit count of parameters of a first machine learning model is decreased in accordance with a bit count of parameters of a second machine learning model (the second AI model M2) in a subsequent stage can be prevented.

[0197] As described with reference to the respective figures of Figs. 1 to 3 or the like, an image input to a second machine learning model (the second AI model M2) may be a crop image Gcr cut out from an input image (for example, a frame image Gfr) as a region containing an object in the inference device 1 (the camera device 1A). Thus, a computation amount of a second inference processing unit (for example, the sensor external control unit 33) can be reduced.

[0198] An inference method according to the present technology performed by a computer device includes: first inference processing to perform inference by a first machine learning model (the first AI model M1, the first AI model M1a for RGB, and the first AI model M1b for YUV) having a parameter quantization bit count of N; and second inference processing to perform inference by a second machine learning model (the second AI model M2) having a parameter quantization bit count of M smaller than N and using an inference result in the first inference processing.

[0199] A program according to the present technology causes an information processing device to perform: first inference processing to perform inference by a first machine learning model (the first AI model M1, the first AI model M1a for RGB, and the first AI model M1b for YUV) having a parameter quantization bit count of N; and second inference processing to perform inference by a second machine learning model (the second AI model M2) having a parameter quantization bit count of M smaller than N and using an inference result in the first inference processing.

[0200] An inference system according to the present technology includes: a first device (for example, the camera device 1A or the image sensor IS) including a first inference processing unit (for example, the AI image processing unit 52) that performs inference by a first machine learning model (the first AI model M1, the first AI model M1a for RGB, and the first AI model M1b for YUV) having a parameter quantization bit count of N; and a second device (for example, the camera device 1A, the information processing device 3, or the server device 2) including a second inference processing unit (for example, the sensor external control unit 33) that performs inference by a second machine learning model (the second AI model M2) having a parameter quantization bit count of M smaller than N and using an inference result by the first inference processing unit.

[0201] The various functions and effects described above can also be obtained by the estimation method, the program, and the inference system.

[0202] Note that the effects described in the present specification are illustrated as an example only and are not limited.　Further, other effects may also be obtained. Further, it is possible to combine the respective examples described above together in any way so long as the combinations are feasible.

[0203] <8. Present Technology> The present technology also makes it possible to employ the following configurations. (1) A non-transitory computer-readable storage medium storing computer-readable instructions thereon which, when executed by an information processing device, cause the information processing device to perform a method, the method comprising: 　　　acquiring an image captured by an image sensor; 　　　detecting, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning model; 　　　cropping the captured image, wherein the cropped image includes the face region; 　　　inputting the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model; 　　　extracting, by the second trained machine learning model, a first feature of the face region in the cropped image; 　　　comparing the first feature with a second feature of a face stored in a memory; 　　　determining whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold; and 　　　in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model. (2) The non-transitory computer-readable storage medium of (1), further comprising: 　　　capturing a plurality of images with an exposure setting changed for each of the plurality of captured images. (3) 　　　The non-transitory computer-readable storage medium of (2), wherein the exposure setting includes changing a brightness of each of the plurality of captured images based on one or more of a shutter speed, an iris setting, or an ISO sensitivity. (4) 　　　The non-transitory computer-readable storage medium of (2), further comprising: 　　　sequentially capturing an intermediate image having an exposure setting representing an average brightness value within a field of a view of a camera comprising the image sensor, an underexposed image having an exposure setting with which underexposure is greater than the intermediate image, and an overexposed image having an exposure setting with which overexposure is greater than the intermediate image. (5) 　　　The non-transitory computer-readable storage medium of (4), wherein regions from which features of a face that are suitably extractable are different for each of a cropped intermediate image, a cropped underexposed image, and a cropped overexposed image. (6) 　　　The non-transitory computer-readable storage medium of (5), further comprising: 　　　integrating extracted features from the cropped intermediate image, the cropped underexposed image, and the cropped overexposed image as an integrated feature. (7) 　　　The non-transitory computer-readable storage medium of (7), further comprising: 　　　comparing the integrated feature with the second feature of the face stored in the memory. (8) 　　　The non-transitory computer-readable storage medium of (7), further comprising: 　　　authenticating the integrated feature as the face stored in the memory in a case that the matching rate of the comparison between the integrated feature and the second feature is greater than or equal to the first predetermined threshold. (9) 　　　The non-transitory computer-readable storage medium of (2), wherein the plurality of captured images includes all images captured within a predetermined amount of time. (10) 　　　The non-transitory computer-readable storage medium of (1), further comprising: 　　　calculating an index value of the cropped image. (11) 　　　The non-transitory computer-readable storage medium of (10), wherein the index value indicates a strength of a blur, a noise amount, or a likelihood of an inference result of the first machine learning model. (12) 　　　The non-transitory computer-readable storage medium of (10), further comprising: 　　　determining whether the index value is greater than or equal to a second predetermined threshold; and 　　　in response to a determination that the index value is greater than or equal to the second predetermined threshold, extracting, by the second trained machine learning model, the first feature of the face region in the cropped image with the second feature of the face stored in the memory. (13) 　　　The non-transitory computer-readable storage medium of (12), further comprising: 　　　in response to a determination that the index value is less than the predetermined threshold, capturing an alternate image to replace the image having the index value less than the predetermined threshold. (14) 　　　An image processing method, comprising: 　　　　　　acquiring an image captured by an image sensor; detecting, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning mode; cropping the captured image, wherein the cropped image includes the face region; inputting the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model; extracting, by the second trained machine learning model, a first feature of the face region in the cropped image; comparing the first feature with a second feature of a face stored in a memory; determining whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold; and in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model. (15) 　　　The method of (14), further comprising: capturing a plurality of images with an exposure setting changed for each of the plurality of captured images. (16) 　　　The method of (15), further comprising: sequentially capturing an intermediate image having an exposure setting representing an average brightness value within a field of a view of a camera comprising the image sensor, an underexposed image having an exposure setting with which underexposure is greater than the intermediate image, and an overexposed image having an exposure setting with which overexposure is greater than the intermediate image. (17) 　　　The method of (16), further comprising: 　　　integrating extracted features from a cropped intermediate image, a cropped underexposed image, and a cropped overexposed image as an integrated feature. (18) 　　　The method of (17), further comprising: 　　　comparing the integrated feature with the second feature of the face stored in the memory. (19) 　　　The method of (18), further comprising: 　　　authenticating the integrated feature as the face stored in the memory in a case that the matching rate of the comparison between the integrated feature and the second feature is greater than or equal to the first predetermined threshold. (20) 　　　An imaging processing system, comprising: 　　　an image sensor configured to capture an image; and 　　　processing circuitry configured to acquire an image captured by an image sensor, detect, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning model, crop the captured image, wherein the cropped image includes the face region, input the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model, extract, by the second trained machine learning model, a first feature of the face region in the cropped image, compare the first feature with a second feature of a face stored in a memory, determine whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold, and in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model. (21) An inference device including: a first inference processing unit that performs inference by a first machine learning model; and a second inference processing unit that performs inference by a second machine learning model using an inference result of the first inference processing unit, wherein a quantization bit count of a parameter used in the first machine learning model is set to be larger than a quantization bit count of a parameter used in the second machine learning model. (22) The inference device according to (21), wherein the first machine learning model is used in processing to infer a region containing an object in an input image, and the second machine learning model is used in processing to recognize the object contained in the region. (23) The inference device according to (22), wherein the first machine learning model is used in processing to infer a face region containing a person’s face in an input image, and the second machine learning model is used in verification between the face contained in the face region and data contained in a database. (24) The inference device according to (23), wherein the second machine learning model is capable of outputting a feature amount of the face contained in the face region, and the second inference processing unit performs person identification by verifying the output feature amount with feature amount data contained in the database. (25) The inference device according to (24), wherein the input image includes a plurality of images having different exposure, and the second inference processing unit performs the person identification using the feature amount for each of the plurality of images. (26) The inference device according to (25), wherein the input image is defined as a frame image captured by an imaging device, the first inference processing unit performs processing to infer the face region for each of the frame images, and the second inference processing unit performs the person identification using the feature amount output from the second machine learning model about the face region for each of a prescribed number of the most recent frame images. (27) The inference device (26), wherein the input image includes an intermediate image defined as being intermediately exposed, a plurality of overexposed images defined as being further overexposed than the intermediate image, and a plurality of underexposed images defined as being further underexposed than the intermediate image, and the second inference processing unit performs the person identification about any of a most overexposed image among the overexposed images and a most underexposed image among the underexposed images after performing the person identification about the intermediate image. (28) The inference device according to (27), wherein the second inference processing unit selects an image to be used in the person identification after the intermediate image on a basis of a brightness value of a region having a small amount of feature extraction in the face region. (29) The inference device according to any of (23) to (28), wherein the second inference processing unit determines whether to input an image of a face region to the second machine learning model according to an index value based on the face region supplied from the first inference processing unit. (30) The inference device according to (29), wherein the index value is defined as an evaluation value about a blur occurring in the face region, and the second inference processing unit determines that an image of the face region is not input to the second machine learning model when the index value indicates occurrence of a blur of a prescribed size or more. (31) The inference device according to any of (29) and (30), wherein the index value is defined as an evaluation value indicating a likelihood of an inference result of the first inference processing unit, and the second inference processing unit determines that the image of the face region concerned is not input to the second machine learning model when the index value is less than a threshold. (32) The inference device according to any of (21) to (31), wherein the inference device is defined as a camera device including an image sensor having a sensor internal control unit as the first inference processing unit and a sensor external control unit that is provided as the second inference processing unit and controls an imaging operation by the image sensor, and an image input to the first machine learning model is defined as a captured image captured by the image sensor. (33) The inference device according to (32), wherein the sensor internal control unit performs processing to infer a face region containing a person’s face in the captured image by inputting the captured image to the first machine learning model, and the sensor external control unit extracts a feature amount of the face contained in the face region by inputting an image of the face region to the second machine learning model, and performs person identification by verifying the feature amount with feature amount data contained in a database. (34) The inference device according to (33), wherein the sensor external control unit transitions to a bracket mode to capture a plurality of images having different exposure when the verification fails, and performs, by the second machine learning model, the person identification using the feature amounts of the faces contained in the face regions in the plurality of images having the different exposure. (35) The inference device according to claim any of (32) to (34), wherein the first machine learning model used by the first inference processing unit is defined as one using a parameter of a quantization bit count corresponding to a data format of the captured image. (36) The inference device according to any of (22) to (35), wherein an image input to the second machine learning model is defined as a crop image cut out from the input image as the region containing the object. (37) An inference method performed by a computer device, the method including: first inference processing to perform inference by a first machine learning model having a parameter quantization bit count of N; and second inference processing to perform inference by a second machine learning model having a parameter quantization bit count of M smaller than N and using an inference result in the first inference processing. (38) A program causing an information processing device to perform: first inference processing to perform inference by a first machine learning model having a parameter quantization bit count of N; and second inference processing to perform inference by a second machine learning model having a parameter quantization bit count of M smaller than N and using an inference result in the first inference processing. (39) An inference system including: a first device including a first inference processing unit that performs inference by a first machine learning model having a parameter quantization bit count of N; and a second device including a second inference processing unit that performs inference by a second machine learning model having a parameter quantization bit count of M smaller than N and using an inference result by the first inference processing unit.

[0204] 1 Inference device 1A Camera device 33 Sensor external control unit (second inference processing unit) 34 Memory unit (database) 43 Sensor internal control unit 52 AI image processing unit (first inference processing unit) IS Image sensor Gfr Frame image Gcr Crop image ARf Face region M1 First AI model (first machine learning model) M1a First AI model for RGB (first machine learning model) M1b First AI model for YUV (first machine learning model) M2 Second AI model (second machine learning model) Md2 Bracket mode Gn Intermediate image Go Overexposed image Gu Underexposed image

Claims

1. A non-transitory computer-readable storage medium storing computer-readable instructions thereon which, when executed by an information processing device, cause the information processing device to perform a method, the method comprising: 　　　acquiring an image captured by an image sensor; 　　　detecting, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning model; 　　　cropping the captured image, wherein the cropped image includes the face region; 　　　inputting the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model; 　　　extracting, by the second trained machine learning model, a first feature of the face region in the cropped image; 　　　comparing the first feature with a second feature of a face stored in a memory; 　　　determining whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold; and 　　　in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model.

2. The non-transitory computer-readable storage medium of claim 1, further comprising: 　　　capturing a plurality of images with an exposure setting changed for each of the plurality of captured images.

3. The non-transitory computer-readable storage medium of claim 2, wherein the exposure setting includes changing a brightness of each of the plurality of captured images based on one or more of a shutter speed, an iris setting, or an ISO sensitivity.

4. The non-transitory computer-readable storage medium of claim 2, further comprising: 　　　sequentially capturing an intermediate image having an exposure setting representing an average brightness value within a field of a view of a camera comprising the image sensor, an underexposed image having an exposure setting with which underexposure is greater than the intermediate image, and an overexposed image having an exposure setting with which overexposure is greater than the intermediate image.

5. The non-transitory computer-readable storage medium of claim 4, wherein regions from which features of a face that are suitably extractable are different for each of a cropped intermediate image, a cropped underexposed image, and a cropped overexposed image.

6. The non-transitory computer-readable storage medium of claim 5, further comprising: 　　　integrating extracted features from the cropped intermediate image, the cropped underexposed image, and the cropped overexposed image as an integrated feature.

7. The non-transitory computer-readable storage medium of claim 6, further comprising: 　　　comparing the integrated feature with the second feature of the face stored in the memory.

8. The non-transitory computer-readable storage medium of claim 7, further comprising: 　　　authenticating the integrated feature as the face stored in the memory in a case that the matching rate of the comparison between the integrated feature and the second feature is greater than or equal to the first predetermined threshold.

9. The non-transitory computer-readable storage medium of claim 2, wherein the plurality of captured images includes all images captured within a predetermined amount of time.

10. The non-transitory computer-readable storage medium of claim 1, further comprising: 　　　calculating an index value of the cropped image.

11. The non-transitory computer-readable storage medium of claim 10, wherein the index value indicates a strength of a blur, a noise amount, or a likelihood of an inference result of the first machine learning model.

12. The non-transitory computer-readable storage medium of claim 10, further comprising: 　　　determining whether the index value is greater than or equal to a second predetermined threshold; and 　　　in response to a determination that the index value is greater than or equal to the second predetermined threshold, extracting, by the second trained machine learning model, the first feature of the face region in the cropped image with the second feature of the face stored in the memory.

13. The non-transitory computer-readable storage medium of claim 12, further comprising: 　　　in response to a determination that the index value is less than the predetermined threshold, capturing an alternate image to replace the image having the index value less than the predetermined threshold.

14. An image processing method, comprising: 　　　　　　acquiring an image captured by an image sensor; detecting, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning mode; cropping the captured image, wherein the cropped image includes the face region; inputting the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model; extracting, by the second trained machine learning model, a first feature of the face region in the cropped image; comparing the first feature with a second feature of a face stored in a memory; determining whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold; and in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model.

15. The method of claim 14, further comprising: capturing a plurality of images with an exposure setting changed for each of the plurality of captured images.

16. The method of claim 15, further comprising: sequentially capturing an intermediate image having an exposure setting representing an average brightness value within a field of a view of a camera comprising the image sensor, an underexposed image having an exposure setting with which underexposure is greater than the intermediate image, and an overexposed image having an exposure setting with which overexposure is greater than the intermediate image.

17. The method of claim 16, further comprising: 　　　integrating extracted features from a cropped intermediate image, a cropped underexposed image, and a cropped overexposed image as an integrated feature.

18. The method of claim 17, further comprising: 　　　comparing the integrated feature with the second feature of the face stored in the memory.

19. The method of claim 18, further comprising: 　　　authenticating the integrated feature as the face stored in the memory in a case that the matching rate of the comparison between the integrated feature and the second feature is greater than or equal to the first predetermined threshold.

20. An imaging processing system, comprising: 　　　an image sensor configured to capture an image; and 　　　processing circuitry configured to 　　　　　　acquire an image captured by an image sensor, detect, by a first trained machine learning model, a face region in the captured image, the first trained machine learning model being a quantized machine learning model, crop the captured image, wherein the cropped image includes the face region, input the cropped image into a second trained machine learning model, the second trained machine learning model being a quantized machine learning model, extract, by the second trained machine learning model, a first feature of the face region in the cropped image, compare the first feature with a second feature of a face stored in a memory, determine whether a matching rate between the first feature and the second feature is greater than or equal to a first predetermined threshold, and in response to a determination that the matching rate is greater than or equal to the first predetermined threshold, performing a face recognition success notification, wherein a first quantization bit count of a parameter used in the first trained quantized machine learning model is set to be larger than a second quantization bit count of a parameter used in the second trained machine learning model.

Images