Face information detection method

Abstract

To appropriately detect face information from a face image showing the face of a driver of a vehicle.SOLUTION: A face information detection method includes: determining whether or not an attachment is attached to a predetermined part of the face of a driver based on a face image showing the face of the driver of a vehicle; if it is determined that no attachment is attached, selecting a first algorithm subjected to learning to detect face information from a face image including no attachment from among a plurality of algorithms for detecting face information based on a form of the face of the driver from the face image; if it is determined that an attachment is attached, selecting a second algorithm from among the plurality of algorithms different from the first algorithm subjected to learning to detect face information from a face image including the attachment; and detecting the face information from the face image using the selected algorithm.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a face information detection method for detecting face information based on the morphology of a driver's face from an image showing the face of a driver of a vehicle. [Background technology]

[0002] The driver whose face is shown in the image may be wearing an attachment such as a mask. Patent Document 1 describes a face direction determination device that determines the face direction of a subject wearing a mask based on the mask distance between the upper and lower ends of the mask worn by the subject. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2022 / 269653 Summary of the Invention [Problem to be solved by the invention]

[0004] The face orientation determination device described in Patent Document 1 performs face orientation determination using a machine learning model that has been trained to be able to detect facial information regardless of whether an image of a face wearing a mask or an image of a face not wearing a mask is input.

[0005] The presence or absence of an attached object may affect the detection of facial information. When a single algorithm (e.g., a machine learning model) is used for detecting facial information regardless of the presence or absence of an attached object, facial information may not be detected properly depending on the presence or absence of an attached object. If facial information cannot be detected properly, it may be difficult to properly implement automatic driving of a vehicle or assistance for driving operations by a driver.

[0006] An object of the present disclosure is to provide a face information detection method capable of appropriately detecting face information from a face image showing the face of a vehicle driver. [Means for solving the problem]

[0007] The gist of the present disclosure is as follows.

[0008] (1) determining whether or not an attachment is attached to a predetermined part of the face of a driver of a vehicle based on a facial image showing the face of the driver of the vehicle; When it is determined that the attachment is not present, a first algorithm is selected from among a plurality of algorithms for detecting facial information based on the shape of the driver's face from the facial image, the first algorithm being trained to detect facial information from a facial image not including an attachment; When it is determined that an attachment is present, a second algorithm different from the first algorithm that is trained to detect face information from a face image including an attachment is selected from the plurality of algorithms; detecting the facial information from the facial image using a selected algorithm; A face information detection method comprising:

[0009] According to the face information detection method of the present disclosure, face information can be appropriately detected from a face image showing the face of a driver of a vehicle. [Brief description of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of a vehicle in which a face information detection device is implemented; [Diagram 2] FIG. 2 is a functional block diagram of a processor included in the face information detection device. [Diagram 3] 13 is a flowchart of a face information detection process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, a face information detection method capable of appropriately detecting face information from a face image showing the face of a vehicle driver will be described in detail with reference to the drawings. In the face information detection method, a face information detection device first determines whether or not an attachment is attached to a predetermined part of the driver's face based on a face image showing the face of the vehicle driver. If it is determined that no attachment is attached, the face information detection device selects a first algorithm learned to detect face information from a face image not including attachments from among a plurality of algorithms for detecting face information based on the shape of the driver's face from a face image. On the other hand, if it is determined that an attachment is attached, the face information detection device selects a second algorithm from among the plurality of algorithms, which is different from the first algorithm learned to detect face information from a face image including attachments. Then, the face information detection device detects face information from the face image using the selected algorithm.

[0012] FIG. 1 is a schematic diagram of a vehicle in which a face information detection device is implemented.

[0013] The vehicle 1 includes a driver monitor camera 2 and a face information detection device 3 .

[0014] The driver monitor camera 2 and the face information detection device 3 are communicatively connected via an in-vehicle network conforming to a standard such as a controller area network.

[0015] The driver monitor camera 2 is an example of a sensor for generating facial images representing the facial area of ​​the driver of the vehicle in a time series. The driver monitor camera 2 has an infrared camera having a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to infrared light, such as CCD or C-MOS, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The driver monitor camera 2 also has a light source that emits infrared light. The driver monitor camera 2 is attached, for example, at the front of the vehicle interior, facing the face of the driver seated in the driver's seat. The driver monitor camera 2 irradiates the driver with infrared light at predetermined time intervals (for example, 1 / 30 to 1 / 10 seconds) and generates facial images showing the driver's face at predetermined time intervals.

[0016] The driver monitor camera 2 may have a visible light camera having a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. In other words, the driver monitor camera may generate at least one of a face image using infrared light and a face image using visible light.

[0017] When the driver monitor camera 2 has an infrared camera and a visible light camera, the imaging optical system of the infrared camera and the imaging optical system of the visible light camera may share at least some optical elements. For example, the imaging optical system of the infrared camera and the imaging optical system of the visible light camera may share a front lens group including a lens arranged closest to the object side, and a beam splitter that emits infrared light of the light emitted from the front lens group toward the two-dimensional detector of the infrared camera and emits visible light toward the two-dimensional detector of the visible light camera. By sharing at least some optical elements between the infrared camera and the visible light camera in this way, the difference in parameters such as the shooting position and shooting direction in the face image by infrared light and the face image by visible light can be reduced.

[0018] The face information detection device 3 is an ECU including a communication interface 31, a memory 32, and a processor 33. The face information detection device 3 detects face information based on the morphology of the driver's face from the face image generated by the driver monitor camera 2.

[0019] The communication interface 31 is an example of a communication unit, and includes a communication interface circuit for connecting the face information detection device 3 to an in-vehicle network. The communication interface 31 supplies the processor 33 with data received from the driver monitor camera 2, for example.

[0020] The memory 32 includes a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 32 stores various data used in processing by the processor 33, such as parameters constituting a determination algorithm for determining the presence or absence of an attachment on a predetermined part of the driver's face based on a facial image. The attachment on a predetermined part of the face is, for example, a mask worn on the mouth area or glasses worn on the eye area.

[0021] The memory 32 also stores parameters that define a number of detection algorithms for detecting facial information from a facial image. The facial information is, for example, information that represents the driver's facial orientation, facial position, or line of sight.

[0022] The memory 32 also stores various application programs, such as a computer program for face information detection that causes the face information detection device 3 to execute a face information detection method.

[0023] The processor 33 is an example of a control unit, and includes one or more processors and their peripheral circuits. The processor 33 may further include other arithmetic circuits, such as a logic arithmetic unit, a numerical arithmetic unit, or a graphics processing unit.

[0024] FIG. 2 is a functional block diagram of the processor 33 included in the face information detection device 3. As shown in FIG.

[0025] The processor 33 of the face information detection device 3 has, as functional blocks, a determination unit 331 and a detection unit 332. Each of these units in the processor 33 is a functional module implemented by a program executed on the processor 33. A computer program that realizes the functions of each unit of the processor 33 may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. Alternatively, each of these units in the processor 33 may be implemented in the face information detection device 3 as an independent integrated circuit, a microprocessor, or firmware.

[0026] The determination unit 331 determines whether or not an attachment is attached to a predetermined part of the driver's face based on the face image acquired from the driver monitor camera 2 via the communication interface 31. The determination unit 331 determines whether or not an attachment is attached to a predetermined part of the driver's face by applying a determination algorithm stored in the memory 32 to the face image.

[0027] The determination algorithm includes, for example, detecting facial feature points such as the inner corners of the eyes, the outer corners of the eyes, and the corners of the mouth from the facial image, identifying an area in the facial image corresponding to a specific part of the face based on the positions of the detected facial feature points, and determining whether or not an article is being worn in the identified area.

[0028] For example, the determination unit 331 detects facial feature points from a facial image by inputting the facial image to a classifier that has been trained in advance to detect facial feature points from the facial image. The classifier can be, for example, a convolutional neural network (CNN) having multiple convolution layers connected in series from the input side to the output side, such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector). A large number of facial images including facial feature points are input to the CNN as training data to perform learning in advance, so that the CNN operates as a classifier that detects facial feature points from a facial image.

[0029] For example, the determination unit 331 compares the positions of the detected facial feature points with a three-dimensional model representing a standard face, and identifies the area of ​​the facial image corresponding to the area around a specific part of the face (e.g., the mouth) in the three-dimensional model when the positions of the facial feature points best match the positions of the detected facial feature points.

[0030] For example, the determination unit 331 obtains and compares representative values ​​of the hues of pixels included in the specified area in the face image and areas other than the specified area among the areas in which the driver's face is represented in the face image. If the value of each pixel included in the face image is defined in RGB (Red, Green, Blue), the determination unit 331 can obtain the hue (H) of each pixel by, for example, converting the pixel value from RGB to HSV (Hue, Saturation, Value) using a predetermined conversion formula.

[0031] The determination unit 331 determines that no attachment is present when the difference between the representative values ​​of the hues corresponding to these regions is smaller than a predetermined color difference threshold, and determines that an attachment is present when the difference is larger than the color difference threshold.

[0032] If the difference between the representative values ​​of the hues corresponding to these regions is greater than a predetermined color difference threshold, the determination unit 331 may further determine the type of the attachment. For example, the determination unit 331 may determine the type of attachment based on the representative values ​​of the hues corresponding to the identified regions.

[0033] The detection unit 332 selects one algorithm from a plurality of algorithms for detecting face information such as face orientation, face position, and gaze direction from a face image, and detects face information from the face image using the selected algorithm.

[0034] Each of the multiple algorithms is, for example, a classifier that has been trained in advance to detect the face information of the driver from a face image. The classifier can be, for example, a CNN. By inputting the face image as training data into the CNN and performing training in advance, the CNN operates as a classifier that detects face information from the face image.

[0035] The plurality of algorithms includes a first algorithm trained to detect facial information from facial images that do not include clothing, and a second algorithm trained to detect facial information from facial images that include clothing.

[0036] The detection unit 332 selects a first algorithm as the one algorithm when it is determined that no attachment is present, and selects a second algorithm when it is determined that an attachment is present.

[0037] The multiple algorithms may further include a third algorithm trained to detect face information from a face image showing an attachment of a first hue (e.g., white) or a fourth algorithm trained to detect face information from a face image showing an attachment of a second hue (e.g., black). In this case, the detection unit 332 selects one algorithm from the multiple algorithms based on the presence or absence of an attachment and the type of the attachment determined by the determination unit 331.

[0038] 3 is a flowchart of the face information detection process. The processor 33 of the face information detection device 3 executes the face information detection process described below at predetermined time intervals (for example, every second) while the vehicle 1 is traveling.

[0039] First, the determination unit 331 of the processor 33 of the face information detection device 3 determines the presence or absence of an attachment attached to a predetermined part of the driver's face based on a face image acquired from the driver monitor camera 2 (step S1).

[0040] If it is determined that no attachment is present (step S1: N), the detection unit 332 of the processor 33 selects a first algorithm from among a plurality of algorithms, the first algorithm being trained to detect facial information from a facial image not including an attachment (step S2).

[0041] If it is determined that an attachment is present (step S1: Y), the detection unit 332 selects, from among the multiple algorithms, a second algorithm that has been trained to detect face information from a face image including an attachment (step S3).

[0042] The detection unit 332 detects face information from the face image using the algorithm selected in step S2 or step S3, and ends the face information detection process.

[0043] By performing the face information detection process in this manner, the face information detection device 3 can appropriately detect face information from a face image showing the face of the driver of the vehicle.

[0044] It should be understood that those skilled in the art can make various changes, substitutions, and alterations thereto without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0045] 1 vehicle 3. Face Information Detection Device 331 Judgment section 332 Detection unit

Claims

[Claim 1] determining whether or not an attachment is attached to a predetermined part of the face of the driver of the vehicle based on a facial image showing the face of the driver of the vehicle; When it is determined that the attachment is not present, a first algorithm is selected from among a plurality of algorithms for detecting facial information based on the shape of the driver's face from the facial image, the first algorithm being trained to detect facial information from a facial image not including an attachment; When it is determined that an attachment is present, a second algorithm different from the first algorithm that is trained to detect face information from a face image including an attachment is selected from among the plurality of algorithms; detecting the facial information from the facial image using a selected algorithm; A face information detection method comprising:

Images