Facial recognition device and facial recognition method

Abstract

The present disclosure relates to a facial recognition device and a facial recognition method. The facial recognition device includes: a camera for capturing a facial image of a driver; a database (DB) for storing facial data including the facial image and one or more environmental conditions corresponding to the facial image; and a controller for extracting facial data from the DB based on environmental conditions at the moment of facial recognition when performing a facial recognition function, and recognizing the driver's face based on the facial image of the extracted facial data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority from Korean Patent Application No. 10-2018-0140193 filed on November 14, 2018, in the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a facial recognition device and a facial recognition method. Background Art

[0004] Recently, vehicles have adopted a function for setting a profile for each driver and performing control according to the setting, such as IMS (Integrated Memory System), USM (User Setup Menu), and the like.

[0005] In order to perform such a function, it is necessary to authenticate the driver. Near Field Communication (NFC) of a smartphone, fingerprint recognition, facial recognition, etc. can be used as a driver authentication method.

[0006] The advantage of facial recognition is that registration and recognition can be performed naturally compared to NFC, fingerprint recognition, etc. of smartphones.

[0007] However, face recognition may have a problem in that recognition performance is degraded due to a face occlusion phenomenon or a change in surrounding environmental factors. Summary of the invention

[0008] The present disclosure aims to solve the above-mentioned problems occurring in the prior art while maintaining the advantages achieved by the prior art.

[0009] An aspect of the present disclosure provides a facial recognition device and a facial recognition method to prevent performance of facial recognition of a driver from being deteriorated due to changes in environmental factors by classifying and storing facial data of a driver according to a plurality of environmental conditions.

[0010] An aspect of the present disclosure provides a face recognition apparatus and a face recognition method that improves the performance of face recognition by calculating a Hamming distance by assigning a weight to each environmental factor and performing face recognition based on face data having similar conditions.

[0011] The technical problems to be solved by the present inventive concept are not limited to the above-mentioned problems, and any other technical problems not mentioned herein will be clearly understood by those skilled in the art to which the present disclosure belongs from the following description.

[0012] According to one aspect of the present disclosure, a facial recognition device includes: a camera that captures a facial image of a driver; a database (DB) that stores facial data including the facial image and one or more environmental conditions corresponding to the facial image; and a controller that, when executing a facial recognition function, extracts facial data from the DB based on the environmental conditions at the moment of recognizing the face, and recognizes the driver's face based on the facial image of the extracted facial data.

[0013] The controller may extract, from among the facial data stored in the DB, facial data of an environmental condition having the highest similarity to the environmental condition at the moment of recognizing the face.

[0014] The facial recognition device may further include: a judgment device, which compares the environmental factors of the environmental conditions included in the facial data with each environmental factor of the environmental conditions at the moment of facial recognition, calculates the Hamming distance of each environmental factor, and judges the similarity based on the calculated Hamming distance of each environmental factor.

[0015] The face recognition device may further include: a conversion device that encodes each environmental factor of the environmental conditions included in the face data and the environmental conditions at the time of recognizing the face.

[0016] The judging means may calculate the Hamming distance by comparing each environmental factor of the code.

[0017] When the codes of the environmental factors corresponding to each other in the environmental conditions included in the facial data and the environmental conditions at the moment of face recognition are the same, the judgment device can calculate the Hamming distance of the corresponding environmental factors as "0", otherwise the Hamming distance can be calculated as "1".

[0018] When the calculated sum of the Hamming distances of each environmental factor is small, the judging means may judge that the similarity is high.

[0019] The judging means may assign a weight to each environmental factor, and judge the similarity by applying the weight to the calculated Hamming distance of each environmental factor.

[0020] The environmental condition may include at least one environmental factor of feature information extracted from the facial image and vehicle status information.

[0021] The facial recognition device may further include: an image analysis device for extracting feature information by analyzing facial images.

[0022] The facial recognition device may further include: a communication device for receiving vehicle status information.

[0023] The controller may acquire the facial image and one or more environmental conditions corresponding to the facial image in real time or at predetermined time intervals during vehicle driving.

[0024] When the environmental conditions corresponding to the facial image and the environmental conditions of the facial data pre-stored corresponding to the driver are different, the controller may further generate facial data including the acquired facial image and the environmental conditions corresponding to the facial image.

[0025] Another aspect of the present disclosure provides a facial recognition method, which includes: photographing a facial image of a driver; storing facial data including the photographed facial image of the driver and one or more environmental conditions corresponding to the facial image in a database (DB); and when executing a facial recognition function, extracting pre-stored facial data from the DB based on the environmental conditions at the moment of recognizing the face, and recognizing the driver's face based on the facial image of the extracted facial data. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other objects, features and advantages of the present disclosure will become more apparent through the following detailed description in conjunction with the accompanying drawings:

[0027] Figure 1 A vehicle to which the facial recognition device according to an embodiment of the present disclosure is applied is shown;

[0028] Figure 2 shows the configuration of a facial recognition device according to an embodiment of the present disclosure;

[0029] Figure 3A and Figure 3B An embodiment referred to in describing the operation of the facial recognition device according to an embodiment of the present disclosure is shown;

[0030] Figure 4A , Figure 4B and Figure 4C Another embodiment of the operation of the facial recognition device is shown;

[0031] Figure 5 , Figure 6 , Figure 7 and Figure 8 Another embodiment of the operation of the facial recognition device is shown;

[0032] Fig.9A and Fig. 9B Another embodiment of the operation of the facial recognition device is shown;

[0033] Fig.10 and Fig.11 is a flowchart of the operation of the method according to an embodiment of the present disclosure; and

[0034] Fig.12 A computing system is shown that performs a method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. When adding reference numerals to the components of each drawing, it should be noted that the same or equivalent components are represented by the same numerals even when shown on other drawings. In addition, when describing the embodiments of the present disclosure, detailed descriptions of well-known features or functions will be excluded so as not to unnecessarily obscure the subject matter of the present disclosure.

[0036] When describing components according to embodiments of the present disclosure, terms such as "first", "second", "A", "B", "(a)", "(b)", etc. may be used. These terms are intended only to distinguish one component from another, and these terms do not limit the nature, sequence, or order of the constituent components. Unless otherwise defined, all terms used herein, including technical terms or scientific terms, have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs. Terms such as those defined in commonly used dictionaries will be interpreted as having the same meanings as the contextual meanings of the relevant art, and will not be interpreted as having ideal or overly formal meanings unless explicitly defined in this application as having such meanings.

[0037] Figure 1 A vehicle to which the facial recognition device according to the present disclosure is applied is shown.

[0038] like Figure 1 As shown, the facial recognition device 100 can obtain the driver's facial image through the camera 130 and register the obtained driver's facial image. In this case, the facial recognition device 100 can confirm the environmental conditions of the vehicle when obtaining the driver's facial image, and register the information about the confirmed environmental conditions in the database (DB) 150 together with the driver's facial image.

[0039] Subsequently, the facial recognition device 100 may extract facial data of an environmental condition similar to the environmental condition at the time of recognizing the driver's face while recognizing the face, and recognize the driver's face based on the extracted facial data.

[0040] The facial recognition device 100 according to the present disclosure may be implemented inside a vehicle. The facial recognition device 100 may be formed integrally with an onboard control unit. On the other hand, the facial recognition device 100 may be implemented as a separate device and may be connected to the control unit of the vehicle through a separate connection device.

[0041] Reference Figure 2 The detailed configuration of the facial recognition device 100 is described in detail in the embodiment of FIG.

[0042] Figure 2 FIG. 2 shows a configuration of a facial recognition device according to an embodiment of the present disclosure. Figure 2, the facial recognition device 100 may include a controller 110, an interface 120, a camera 130, a communication device 140, a database (DB) 150, an image analysis device 160, a conversion device 170, and a judgment device 180. In the present disclosure, the controller 110, the image analysis device 160, the conversion device 170, and the judgment device 180 of the facial recognition device 100 according to the present embodiment may be implemented as at least one processor.

[0043] The controller 110 may process signals transmitted between components of the facial recognition apparatus 100 .

[0044] The interface 120 may include an input device for receiving a control instruction from the driver and an output device for outputting an operation state, a result, etc. of the facial recognition device 100 .

[0045] In this case, the input device may include key buttons, and may also include a mouse, a joystick, a shuttle knob, a stylus pen, etc. In addition, the input device may also include soft keys implemented on the display.

[0046] The output device may include a display, and may also include an audio output device such as a speaker. In this case, when a touch sensor such as a touch film, a touch sheet, or a touch pad is provided in the display, the display may be operated as a touch screen, and may be implemented in a form in which the input device and the output device are integrated.

[0047] As an example, the display may display a guide screen for registering the driver's facial data. In addition, the display may display a screen showing the facial recognition result.

[0048] The display may include at least one of a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFTLCD), an organic light emitting diode (OLED), a flexible display, a field emission display (FED), and a three-dimensional display (3D display).

[0049] The camera 130 may be a device for capturing the driver's facial image. The camera 130 may be disposed in front of the driver in the vehicle. The camera 130 may continuously or periodically capture the driver's facial image when the vehicle is started. The camera 130 may transmit the captured facial image to the controller 110.

[0050] The communication device 140 may include a communication module that supports a communication interface with an electrical component and / or a control unit provided in the vehicle. As an example, the communication module may receive information about the opening / closing state of a window, the opening / closing state of a sunroof, and / or time zone information from the control unit.

[0051] In the present disclosure, the communication module may include a module supporting vehicle network communication such as CAN (Controller Area Network) communication, LIN (Local Interconnect Network) communication, and Flex-Ray communication.

[0052] In addition, the communication module may include a module for wireless Internet connection or a module for short-range communication. In the present disclosure, wireless Internet technology may include wireless LAN (WLAN), wireless broadband (WIBRO), Wi-Fi, World Interoperability for Microwave Access (WIMAX), etc., and short-range communication technology may include Bluetooth, ZigBee, ultra-wideband (UWB), radio frequency identification (RFID), infrared data communication (IrDA), etc.

[0053] The DB 150 may store data and / or algorithms required for the facial recognition apparatus 100 to operate.

[0054] DB 150 may store instructions and / or algorithms for performing facial recognition operations. In addition, DB 150 may store multiple pieces of facial data registered for each driver. In this case, multiple pieces of facial data may be classified and stored according to the environmental conditions included in the facial data.

[0055] In the present disclosure, the DB 150 may include storage media such as random access memory (RAM), static random access memory (SRAM), read only memory (ROM), programmable read only memory (PROM), electrically erasable programmable read only memory (EEPROM), etc.

[0056] When receiving a facial image of the driver captured by the camera 130 according to a new registration request for the driver's face, the controller 110 may transmit the received facial image to the image analyzing device 160 .

[0057] The image analyzing device 160 can analyze facial images and extract feature information.

[0058] As an example, the image analyzing device 160 may extract size information of the mouth and / or eyes from the facial image, and extract feature information about whether the facial expression represents a neutral expression or a smiling expression based on the extracted size information of the mouth and / or eyes.

[0059] In addition, the image analyzing device 160 may extract feature information about whether the facial direction is forward, left, or right from the facial image.

[0060] In addition, the image analysis device 160 can extract feature information about whether the driver wears accessories such as sunglasses and / or a mask from the facial image.

[0061] The image analyzing device 160 may analyze the facial image and transmit the extracted at least one piece of feature information to the controller 110 .

[0062] Furthermore, the controller 110 may receive the status information of the vehicle at the time when the facial image of the driver is captured from the communication device 140 .

[0063] The controller 110 may generate facial data including one or more environmental conditions corresponding to the facial image and the facial image. The one or more environmental conditions corresponding to the facial image may include a plurality of environmental factors, i.e., feature information extracted from the facial image and status information of the vehicle received from the communication device 140. In the following description, at least one environmental condition corresponding to the facial image is referred to as a "first environmental condition". The controller 110 may store the generated facial data in the DB 150.

[0064] The controller 110 may acquire the facial image and the first environmental condition in real time or at predetermined time intervals during the driving of the vehicle. When the acquired first environmental condition is different from the environmental condition of the facial data stored in the DB 150, the controller 110 may further generate facial data including the acquired facial image and the first environmental condition, and store the facial data in the DB 150.

[0065] Therefore, DB 150 may store a plurality of pieces of face data including a face image of a driver photographed at each moment and a first environmental condition of different environmental factors corresponding to the face image. On the other hand, DB 150 may store face data corresponding to two or more drivers.

[0066] The controller 110 may configure a guide screen during the driver face registration process and display the guide screen on the display screen of the interface 120 .

[0067] Will refer to Figure 3A , Figure 3B , Figure 4A , Figure 4B , Figure 4C and Figure 5 An embodiment of a guidance screen displayed during a face registration process is described.

[0068] Figure 3A and Figure 3B A start guidance screen for the face registration process is shown. Figure 4A , Figure 4B , Figure 4C and Figure 5 A progress guidance screen showing the face registration process.

[0069] When there is a new registration request for the driver's face, such as Figure 3AAs shown, the controller 110 may configure a first guide screen including a message requesting the driver to look at the camera 130 and display the first guide screen on the display screen. Then, when the driver looks at the camera 130, as shown in FIG. Figure 3B As shown, the controller 110 may configure a second guide screen for illustrating an operation for capturing a facial image and display the second guide screen on the display screen.

[0070] When the facial image is started to be captured through the first guide screen and the second guide screen, as shown in FIG. FIG. 4A to FIG. 4C As shown, the controller 110 can provide the registration progress rate through the third to fifth guide screens. When the driver's face registration process is completed, as shown in FIG. Figure 5 As shown, the controller 110 may be configured to display a sixth guide screen showing that the registration is completed and display the sixth guide screen on the display screen.

[0071] The first to sixth guide screens may include a guide message, an emoticon, and / or a status bar, etc. The guide screen may further include means for showing a progress status of the face registration process.

[0072] The controller 110 may classify the plurality of pieces of facial data corresponding to the driver stored in the DB 150 according to each environmental factor of the first environmental condition included in each piece of facial data. Figure 6 Embodiments for classifying facial data are described.

[0073] Figure 6 The face data classified according to each environmental factor of the first environmental condition is shown. Figure 6 , each piece of facial data may include a facial image taken at each moment.

[0074] In addition, each piece of facial data may include a first environmental condition acquired at the time of capturing the facial image. In this case, the first environmental condition may include environmental factors corresponding to feature information such as facial expression, facial direction, whether the driver wears sunglasses, and / or whether the driver wears a mask. In addition, the first environmental condition may include environmental factors corresponding to status information such as whether the car window is open or closed, whether the sunroof is open, and / or the time zone.

[0075] Figure 6 The first facial data among the multiple facial data shown in may include environmental factors corresponding to the unopened state of the car window, the unopened state of the sunroof, daytime, no expression, the face direction is forward, not wearing sunglasses, and not wearing a mask.

[0076] The second facial data may include environmental factors corresponding to a car window not being opened, a sunroof not being opened, daytime, no expression, a face facing right, not wearing sunglasses, and / or not wearing a mask.

[0077] The third facial data may include environmental factors corresponding to the open state of the car windows, the closed state of the sunroof, daytime, smiling expression, face direction forward, not wearing sunglasses, and not wearing a mask.

[0078] The fourth facial data may include environmental factors corresponding to the unopened state of the car window, the unopened state of the sunroof, daytime, no expression, the face direction forward, not wearing sunglasses, and wearing a mask.

[0079] The fifth facial data may include environmental factors corresponding to the closed state of the car window, the open state of the sunroof, night, no expression, the face direction forward, wearing sunglasses and not wearing a mask.

[0080] like Figure 6 As shown, the first to fifth pieces of facial data are different from each other in at least one environmental factor.

[0081] Therefore, the face recognition device 100 according to the present disclosure may improve the performance of face recognition according to each environmental factor by using the face data of the driver classified according to each environmental factor of the first environmental condition.

[0082] When the facial recognition function is executed, the controller 110 can obtain the facial image at the moment of executing the facial recognition function and the environmental conditions at the moment of recognizing the face. In the following description, the environmental conditions at the moment of recognizing the face are referred to as the second environmental conditions. In this case, the controller 110 can compare the environmental factors of the second environmental conditions with each environmental factor of the first environmental conditions included in the multiple facial data stored in DB150, and extract the facial data of the environmental conditions with the highest similarity to the second environmental conditions. The controller 110 can recognize the driver's face based on the facial image of the extracted facial data.

[0083] The controller 110 may control the operations of the conversion device 170 and the judgment device 180 to extract the facial data of the environmental condition having the highest similarity to the second environmental condition.

[0084] First, the conversion device 170 may encode each environmental factor of the first environmental condition and the second environmental condition of the facial data stored in the DB 150 according to a request of the controller 110 .

[0085] Will refer to Figure 7 An embodiment for encoding each environmental factor is described.

[0086] Figure 7 An embodiment of encoding the environmental factor of the first environmental condition included in each piece of facial data stored in DB 150 is shown. The conversion device 170 may encode "O" as "1" and encode "X" as "0". Moreover, the conversion device 170 may encode the "daytime" time zone as "0" and encode the "nighttime" time zone as "1". In addition, the conversion device 170 may encode the facial expression corresponding to "no expression" as "0", and encode the facial expression corresponding to "smiling expression" as "1". In addition, the conversion device 170 may encode the "forward" facial direction as "0", the "left" facial direction as "1", and the "right" facial direction as "2".

[0087] Based on the above conditions, if Figure 7 As indicated by reference numeral 711 in FIG. 1 , the conversion device 170 may encode each environmental factor of the first piece of facial data as “0000000”. Figure 7 As indicated by reference numeral 712 in FIG. 1 , the conversion device 170 may encode the environmental factor of the second piece of facial data as “0000200”. Figure 7 As indicated by reference numeral 713 in FIG. 1 , the conversion device 170 may encode the environmental factor of the third piece of facial data as “1001000”.

[0088] The judgment device 180 may compare the code corresponding to each environmental factor of the first environmental condition included in one or more pieces of facial data with the code corresponding to each environmental factor of the second environmental condition to calculate the Hamming distance. In this case, the judgment device 180 may calculate the Hamming distance between the first environmental condition and the second environmental condition of the facial data based on the value obtained by summing the Hamming distances of each environmental factor.

[0089] exist Figure 7 In the embodiment of FIG. 1 , when it is assumed that the code of each environmental factor of the second environmental condition is “0000000”, as Figure 7 As indicated by the reference mark 715 in the figure, the value "0" is calculated by summing the Hamming distances of the environmental conditions of the first facial data and the environmental factors of the second environmental conditions, the value "1" is calculated by summing the Hamming distances of the environmental conditions of the second facial data and the environmental factors of the second environmental conditions, and the value "2" is calculated by summing the Hamming distances of the environmental conditions of the third facial data and the environmental factors of the second environmental conditions.

[0090] Will refer to Figure 8 An embodiment for calculating the Hamming distance is described.

[0091] Figure 8An embodiment for calculating the Hamming distance based on the code of each environmental factor of the first environmental condition and the second environmental condition is shown. Figure 8 , No.1 represents the code of each environmental factor of the second environmental condition, and No.2 represents the code of each environmental factor of the first environmental condition included in the second facial data.

[0092] In this case, when it is assumed that the code of the environmental factor of the second environmental condition is "0000000" and the code of the environmental factor of the first environmental condition is "0000200", as Figure 8 As indicated by reference numeral 811 in FIG. 1 , the judging device 180 may calculate the value as “0” when the codes of the corresponding environmental factors in the first environmental condition and the second environmental condition are the same and “0”, and Figure 8 As indicated by reference numeral 812 in FIG. 1 , the determining device 180 may calculate “1” when the codes of the corresponding environmental factors in the first environmental condition and the second environmental condition are different from each other.

[0093] In this way, the Hamming distance of each environmental factor of the first environmental condition and the second environmental condition is calculated, and in this case, as indicated by reference numeral 813, the value obtained by summing the Hamming distance of each environmental factor is "1" (for example, 0+0+0+0+1+0+0=1).

[0094] The determination device 180 may determine the similarity based on the calculated Hamming distance. In this case, the smaller the sum of the calculated Hamming distances of each environmental factor is, the higher the similarity between the two environmental conditions may be determined by the determination device 180.

[0095] Meanwhile, the judging device 180 may assign a weight to each environmental factor of the environmental condition, and may judge the similarity by applying the weight to the calculated Hamming distance of each environmental factor.

[0096] Reference Fig.9A and Fig. 9B An embodiment for assigning a weight to each environmental factor of an environmental condition and calculating a Hamming distance by applying the weight is described.

[0097] Fig.9A An embodiment for assigning a weight to each environmental factor of an environmental condition is shown, Fig. 9B An embodiment is shown in which the Hamming distance is calculated by applying weights.

[0098] Environmental factors that affect facial recognition can be given greater weight based on changes in conditions. For example, whether the sunroof is open or closed may have a greater impact on facial recognition than whether the windows are open or closed. In addition, whether a mask is worn may have a great impact on facial recognition. Therefore, if Fig.9AAs shown, the judgment device 180 can assign weights "1" to "2" to environmental factors corresponding to the opening / closing of the vehicle windows, assign weights "3" to "4" to environmental factors corresponding to the opening of the sunroof, assign weights "1" to "2" to environmental factors corresponding to the time zone, assign weight "2" to environmental factors corresponding to facial expressions, facial directions, and whether or not sunglasses are worn, and assign weight "3" to environmental factors corresponding to whether or not a mask is worn.

[0099] In this case, weights corresponding to environmental factors such as whether a window affected by changes in light environment is open or closed, whether a sunroof is open, and a time zone may be subdivided and assigned based on the camera exposure value instead of a fixed value.

[0100] As an example, when the camera exposure value is less than a reference value of an environmental factor corresponding to whether the window is open or closed, a weight of "1" is assigned, and when the camera exposure value is equal to or greater than the reference value, a weight of "2" may be assigned.

[0101] like Fig.9A As shown, when a weight is assigned to each environmental factor in the environmental condition, the judgment device 180 can calculate the Hamming distance between the first environmental condition and the second environmental condition based on the sum of the values ​​obtained by multiplying the Hamming distance calculated for each environmental factor by the weight.

[0102] When Figure 7 When no weight is assigned, the Hamming distance calculated for the environmental condition of the second facial data is "1", while Fig. 9B When the weights are assigned as shown, the Hamming distance calculated for the environmental conditions of the second facial data is "2".

[0103] Therefore, the judgment device 180 can more accurately judge the similarity between the first environmental condition and the second environmental condition of each facial data by assigning weights.

[0104] In this case, the controller 110 may extract facial data having the highest similarity to the second environmental condition according to the judgment result of the judgment device 180. As an example, the controller 110 may extract facial data having a Hamming distance of "0" compared to the second environmental condition. On the other hand, when there is no facial data having a Hamming distance of "0" compared to the second environmental condition, the controller 110 may extract facial data having the shortest Hamming distance. Therefore, the controller 110 may recognize the driver's face based on the facial image of the extracted facial data.

[0105] When the extracted facial data includes a plurality of facial data, the controller 110 may perform facial recognition of the driver based on facial data with a low weight of the Hamming distance applied to each environmental factor among the extracted plurality of facial data. In this case, since a greater weight may be given to the environmental factor when the environmental factor has a greater influence on facial recognition, facial recognition performance may be improved when the facial data has a lower weight applied to the Hamming distance.

[0106] The face recognition device 100 according to the embodiment operated as above may be implemented in the form of an independent hardware device including a memory and a processor processing each operation, and may operate in the form included in another hardware device such as a microprocessor or a general-purpose computer system.

[0107] The operation flow of the facial recognition device according to the present disclosure configured as above is described in detail below.

[0108] Fig.10 4 is a flowchart of a face registration method of a face recognition device according to an embodiment of the present disclosure.

[0109] Reference Fig.10 When the new registration function of the face is performed according to the driver's request (S110), the facial recognition device 100 can obtain the facial image of the driver captured by the camera 130 (S120). In this case, the facial recognition device 100 can confirm the first environmental condition at the time of capturing the facial image based on the facial image acquired in step S120 and the information received through the communication device 140 (S130).

[0110] The facial recognition device 100 may classify the facial data registered in the DB 150 according to the first environmental condition included in each piece of facial data, and compare the first environmental condition confirmed in step S130 with the facial data pre-registered in the DB 150 (S140).

[0111] When the first environmental condition confirmed in step S130 corresponds to a new condition not included in the pre-registered facial data ( S150 ), the facial recognition apparatus 100 may add the facial data of the driver including the facial image and the first environmental condition to the DB 150 ( S160 ).

[0112] The steps from S120 to S160 may be repeatedly performed until the driving of the vehicle ends. When the driving of the vehicle ends (S170), the facial recognition device 100 may end the relevant operation.

[0113] Fig.11 A flowchart showing a face recognition method of a face recognition device according to an embodiment of the present disclosure is shown.

[0114] Reference Fig.11 When the facial recognition function is executed ( S210 ), the facial recognition device 100 may confirm a second environmental condition at the time of executing the facial recognition function ( S220 ).

[0115] The facial recognition device 100 may compare the second environmental condition confirmed in step S220 with the first environmental condition included in the plurality of pieces of facial data stored in the DB 150, and extract facial data having a condition similar to the second environmental condition (S230). Figures 7 to 9A The embodiment gives a specific description of the operation of extracting facial data having conditions similar to the second environmental conditions.

[0116] Thereafter, the facial recognition device 100 may perform a facial recognition operation based on the facial image of the facial data extracted in step S230 ( S240 ).

[0117] When the facial recognition of the driver in step S240 is completed (S250), the facial recognition device 100 can end the relevant operation. On the other hand, when the facial recognition of the driver in step S240 is not completed, the facial recognition device 100 can notify the user of the incomplete state of facial recognition (S260) and perform the steps after S210 again.

[0118] Fig.12 A computing system that executes the method according to an embodiment of the present disclosure is shown. Fig.12 , the computing system 1000 may include at least one processor 1100 , a memory 1300 , a user interface input device 1400 , a user interface output device 1500 , a storage device 1600 , and a network interface 1700 connected to each other via a bus 1200 .

[0119] The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and / or the storage device 1600. The memory 1300 and the storage device 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a ROM (Read Only Memory) 1310 and a RAM (Random Access Memory) 1320.

[0120] Therefore, the operation of the method or algorithm described in conjunction with the embodiments disclosed herein can be directly embodied in hardware, software modules, or a combination thereof executed by the processor 1100. The software module can reside in a storage medium (i.e., memory 1300 and / or storage device 1600) such as RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, register, hard disk, removable disk, CD-ROM. An exemplary storage medium can be connected to the processor 1100, and the processor 1100 can read information from the storage medium and can record information in the storage medium. Alternatively, the storage medium can be integrated with the processor 1100. The processor 1100 and the storage medium can reside in an application specific integrated circuit (ASIC). The ASIC can reside in a user terminal. In another case, the processor 1100 and the storage medium can reside in a user terminal as a separate component.

[0121] Therefore, the exemplary embodiments of the present disclosure are provided to explain rather than to limit the idea and scope of the present disclosure, so that the idea and scope of the present disclosure are not limited by these embodiments. The scope of the present disclosure should be interpreted based on the attached claims, and all technical concepts within the scope equivalent to the claims should be included in the scope of the present disclosure.

[0122] According to the present disclosure, it is possible to prevent the performance of facial recognition of a driver from being deteriorated due to changes in environmental factors by classifying and storing facial data of the driver according to a plurality of environmental conditions.

[0123] In addition, according to the present disclosure, it is possible to improve the performance of facial recognition by calculating the Hamming distance by assigning a weight to each environmental factor and performing facial recognition based on facial data having similar conditions.

[0124] Although the present disclosure has been described above with reference to the exemplary embodiments and the accompanying drawings, the present disclosure is not limited thereto but may be variously modified and changed by those skilled in the art without departing from the idea and scope of the present disclosure as claimed in the claims.

Claims

1. A facial recognition device, include: A camera to capture the driver’s facial image; a database, namely DB, storing facial data including the facial image and one or more environmental conditions corresponding to the facial image; a controller that, when executing a facial recognition function, extracts facial data from the DB based on environmental conditions at the time of recognizing a face, and recognizes the driver's face based on a facial image of the extracted facial data; and a judging device that compares the environmental factors of the environmental conditions included in the facial data with each environmental factor of the environmental conditions at the time of face recognition, calculates the Hamming distance of each environmental factor, and judges the similarity based on the calculated Hamming distance of each environmental factor, The smaller the calculated sum of the Hamming distances of each environmental factor is, the higher the similarity determined by the judgment device is.

2. The facial recognition device according to claim 1, in, The controller extracts facial data of environmental conditions having the highest similarity to the environmental conditions at the time of recognizing the face from among the facial data stored in the DB.

3. The facial recognition device according to claim 1, further comprising: include: a conversion device for encoding each environmental factor of the environmental conditions included in the facial data and the environmental conditions at the time of recognizing the face, The judging device calculates the Hamming distance by comparing each of the encoded environmental factors.

4. The facial recognition device according to claim 3, in, When the codes of the environmental factors corresponding to each other in the environmental conditions included in the facial data and the environmental conditions at the moment of face recognition are the same, the judgment device calculates the Hamming distance of the corresponding environmental factors as "0", otherwise the Hamming distance is calculated as "1".

5. The facial recognition device according to claim 1, in, The judgment means assigns a weight to each of the environmental factors, and judges the similarity by applying the weight to the calculated Hamming distance of each of the environmental factors.

6. The facial recognition device according to claim 5, in, When the extracted facial data includes a plurality of pieces of facial data, the controller performs facial recognition of the driver based on facial data having a low weight of a Hamming distance applied to each environmental factor among the extracted facial data.

7. The facial recognition device according to claim 1, in, The environmental condition includes at least one environmental factor of feature information extracted from the facial image and vehicle status information.

8. The facial recognition device according to claim 7, further comprising: include: The image analysis device extracts the feature information by analyzing the facial image.

9. The facial recognition device according to claim 7, further comprising: include: The communication device receives the vehicle status information.

10. The facial recognition device according to claim 1, in, The controller acquires the facial image and one or more environmental conditions corresponding to the facial image in real time or at predetermined time intervals during vehicle driving.

11. The facial recognition device according to claim 1, in, When the environmental conditions corresponding to the facial image and the environmental conditions of the facial data pre-stored corresponding to the driver are different, the controller further generates facial data including the acquired facial image and the environmental conditions corresponding to the facial image.

12. A facial recognition method, include: Take a picture of the driver's face; storing facial data including the captured facial image of the driver and one or more environmental conditions corresponding to the facial image in a database (DB); When the facial recognition function is executed, the pre-stored facial data is extracted from the DB based on the environmental conditions at the time of facial recognition, and the driver's face is recognized based on the facial image of the extracted facial data; comparing the environmental factors of the environmental conditions included in the facial data with each environmental factor of the environmental conditions at the time of the face recognition, and calculating the Hamming distance of each environmental factor; and The similarity is determined based on the calculated Hamming distance of each environmental factor. The smaller the calculated sum of the Hamming distances of each environmental factor is, the higher the similarity is judged.

13. The facial recognition method according to claim 12, in, Identifying the driver's face includes: determining the similarity between the environmental conditions included in the facial data stored in the DB and the environmental conditions at the time of recognizing the face; and The facial data stored in the DB is extracted from the facial data having the highest similarity to the environmental conditions at the time of the face recognition.

14. The facial recognition method according to claim 12, in, Calculating the Hamming distance for each environmental factor involves: encoding each environmental factor of the environmental conditions included in the facial data and the environmental conditions at the time of the face recognition; and The Hamming distance is calculated by comparing each of the environmental factors of the encoding.

15. The facial recognition method according to claim 14, in, Calculating the Hamming distance for each environmental factor involves: When the codes of the environmental factors corresponding to each other in the environmental conditions included in the facial data and the environmental conditions at the moment of face recognition are the same, the Hamming distance of the corresponding environmental factors is calculated as "0", otherwise the Hamming distance is calculated as "1".

16. The facial recognition method according to claim 12, in, Determining the similarity further includes: assigning a weight to each of said environmental factors; and The similarity is determined by applying the weight to the calculated Hamming distance of each of the environmental factors.

17. The facial recognition method according to claim 16, in, Recognizing the driver's face includes: When the extracted facial data includes a plurality of pieces of facial data, facial recognition of the driver is performed based on facial data having a low weight of a Hamming distance for each environmental factor applied to the extracted facial data.

18. The facial recognition method according to claim 12, in, The environmental condition includes at least one environmental factor of feature information extracted from the facial image and vehicle status information.

19. The facial recognition method according to claim 18, further comprising: include: The feature information is extracted by analyzing the facial image.

20. The facial recognition method according to claim 18, further comprising: include: The vehicle status information is received.

21. The facial recognition method according to claim 12, further comprising: include: The facial image and one or more environmental conditions corresponding to the facial image are acquired in real time or at predetermined time intervals during vehicle driving.

22. The facial recognition method according to claim 12, further comprising: include: When the environmental conditions corresponding to the facial image and the environmental conditions of the facial data pre-stored corresponding to the driver are different, facial data including the acquired facial image and the environmental conditions corresponding to the facial image are further generated.

Images