Facial recognition system, facial recognition program, and facial recognition method

Abstract

This invention provides a facial recognition system, a facial recognition program, and a facial recognition method for appropriately performing authentication processing for online learning. [Solution] A facial recognition system for performing facial recognition of online learning participants, wherein the information processing device 1 comprises an acquisition unit 12, a tracking processing unit 13, and an authentication processing unit 15, and the authentication device 4 comprises an authentication database 5, the authentication database stores authentication facial information for authenticating participants, the acquisition unit acquires the participant's facial information while they are taking the online learning course, the tracking processing unit performs facial tracking based on the facial information, and the authentication processing unit performs authentication based on the facial information from which the facial tracking process was successful and the authenticated facial information.

Description

Technical Field

[0006] , , , ,

[0005] , , , ,

[0001] The present invention relates to a face authentication system, a face authentication program, and a face authentication method.

Background Art

[0002] In recent years, with the digitization of the lecture environment and educational sites, e-learning has been rapidly spreading. On the other hand, with the spread of such lecture systems, there has arisen a problem that the number of examinees who engage in improper behavior has increased, and the importance of face authentication for preventing such improper behavior has been increasing. An example of such face authentication technology is proposed in, for example, Patent Document 1.

[0003] For example, Patent Document 1 describes that even when the examinee is located in a remote area, by processing the personal identification image pre-registered as the face image of the examinee so as to conform to the personal identification process for confirming that the examinee taking the test during the test is the person himself / herself, it is possible to perform personal identification and conduct the test while ensuring that the answerer is the person himself / herself.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in Patent Document 1, there are cases where personal identification cannot be performed even for natural posture changes of e-learning examinees during the test, and there has arisen a problem that even if no improper behavior is being committed, it is determined that improper behavior is being committed, which is very inconvenient for the examinees.

[0006] In view of the above problems, an object of the present invention is to provide a technology capable of appropriately performing authentication processing for online learning. [Means for solving the problem]

[0007] To solve the above problems, the present invention provides a facial recognition system for performing facial recognition of online learning participants, wherein the facial recognition system comprises a database, an acquisition unit, a tracking processing unit, and an authentication processing unit, the database stores authentication facial information for authenticating the participant, the acquisition unit acquires the participant's facial information displayed during online learning, the tracking processing unit performs facial tracking processing based on the facial information, and the authentication processing unit performs authentication processing based on the facial information obtained from the successful facial tracking processing and the authentication facial information.

[0008] Furthermore, in order to solve the above problems, the present invention provides a facial recognition program for performing facial recognition of online learning participants, wherein a computer functions as a database, an acquisition unit, a tracking processing unit, and an authentication processing unit, the database stores authentication facial information for authenticating the participant, the acquisition unit acquires the participant's facial information displayed during online learning, the tracking processing unit performs facial tracking processing based on the facial information, and the authentication processing unit performs authentication processing based on the facial information obtained from the successful facial tracking process and the authentication facial information.

[0009] Furthermore, in order to solve the above problems, the present invention provides a facial recognition method for performing facial recognition of an online learning participant, wherein a computer performs the following processes: storing authentication facial information for authenticating the participant; acquiring the participant's facial information displayed during the online learning session; performing a facial tracking process based on the facial information; and performing an authentication process based on the facial information obtained from the successful facial tracking process and the authentication facial information.

[0010] This configuration significantly reduces authentication failures. This allows for both the prevention of fraudulent activity and a stress-free online learning experience.

[0011] In a preferred embodiment of the present invention, the tracking processing unit determines to stop the online learning if the face tracking process fails a predetermined number of times, and the authentication processing unit determines to resume the stopped online learning if the authentication process is successful.

[0012] This configuration prevents unauthorized online learning by avoiding unnecessary facial movements that could trigger the authentication process.

[0013] In a preferred embodiment of the present invention, the facial recognition system further comprises an image adjustment unit, the image adjustment unit scales the facial information based on the facial information obtained from the successful facial tracking process, and the authentication processing unit performs an authentication process based on the scaled facial information and the authenticated facial information.

[0014] This configuration can reduce the processing load required for authentication. For example, by adjusting the size and aspect ratio of the facial image data to remove unnecessary parts of the image data, the file size of the facial image data can be reduced, thereby reducing the processing load required for authentication.

[0015] In a preferred embodiment of the present invention, the facial recognition system further comprises an image adjustment unit, which identifies areas to be authenticated and areas to be not authenticated in the facial information based on the facial contour of the participant included in the facial information, and adjusts the areas to be not authenticated.

[0016] This configuration improves the accuracy of authentication processing based on the domain being authenticated.

[0017] In a preferred embodiment of the present invention, the image adjustment unit adjusts the area to be authenticated and the visual characteristic values ​​of the area to be authenticated so that the visual characteristic ratio based on the visual characteristic values ​​of the area to be authenticated and the visual characteristic values ​​of the area not to be authenticated becomes a predetermined ratio.

[0018] By adopting such a configuration, the accuracy of the authentication process for the authentication target area can be improved.

[0019] In a preferred form of the present invention, the authentication processing unit executes an authentication process based on the face feature amounts of a plurality of the face information for which the face tracking process has failed and the face feature amount of the authentication face information.

[0020] By adopting such a configuration, the frequency of stopping online learning due to the failure of the face tracking process can be reduced. As a result, it becomes possible to reuse a plurality of face information for which the face tracking process has failed, and the stress of the learners can be further reduced.

Advantages of the Invention

[0021] The present invention has an effect of providing a technology capable of appropriately performing an authentication process for online learning.

Brief Description of the Drawings

[0022] [Figure 1] It is a block diagram showing the configuration of a system in an embodiment of the present invention. [Figure 2] It is a hardware configuration diagram of a system in an embodiment of the present invention. [Figure 3] It is a functional block diagram of a device in an embodiment of the present invention. [Figure 4] It is a processing flowchart of a system in an embodiment of the present invention. [Figure 5] It is a detailed flowchart of the face tracking process in an embodiment of the present invention. [Figure 6] It is a processing flowchart of a face authentication system in conventional online learning.

Modes for Carrying Out the Invention

[0023] Further details will be provided below with reference to the attached drawings. The drawings show preferred embodiments. However, many different forms are possible and the embodiments are not limited to those described herein.

[0024] For example, in this embodiment, the configuration and operation of the facial recognition system will be described, but similar methods, devices, computer programs, etc., can achieve similar effects. Furthermore, the program may be stored on a recording medium or made available for download from an external server. Using this recording medium, for example, the program can be installed on a computer, thereby configuring a facial recognition device and a facial recognition system. Here, the recording medium on which the program is stored may be a non-transient recording medium such as a CD-ROM.

[0025] <1. Overview of the Invention> This invention relates to a facial recognition system for performing facial recognition of learners in online learning. Conventional facial recognition systems have often used a method (Figure 6) in which facial images are acquired at predetermined intervals intended to deter fraud and authentication is performed each time. However, this method has the problem that the learner's face may move out of the front of the webcam due to changes in posture that occur naturally while the learner is referring to textbooks or notes, or while thinking. As a result, authentication errors occur frequently, not only interrupting the learning process often but also increasing the number of times re-authentication is required. Furthermore, this problem can cause stress for learners and may lead to a decrease in learning efficiency.

[0026] Furthermore, a further problem is that while conventional facial recognition systems often suspect fraudulent participation after a certain number of consecutive authentication errors, this method has difficulty clearly distinguishing between temporary authentication errors due to legitimate reasons such as natural changes in posture during thinking, and fraudulent acts such as intentionally leaving one's seat during a lecture and not listening. As a result, there is a risk of misidentification where legitimate actions are deemed fraudulent, and of fraud going unnoticed.

[0027] To address these problems, the present invention employs a two-stage process in its authentication process: a face tracking process and an authentication process. First, in the face tracking process, the present invention acquires the facial information of the student during online learning via the student terminal device of the student using a browser, and determines whether the facial information has been accurately captured. If the facial information has been accurately captured, the authentication process then uses the facial information to perform identity authentication, thereby determining whether the student is taking the course without engaging in fraudulent activity.

[0028] <1.1. System Configuration> Figure 1 is a block diagram showing the system configuration of the present invention. As shown in Figure 1, the facial recognition system 0 comprises an information processing device 1, a student terminal device 3, and an authentication device 4. In this embodiment, the information processing device 1, the student terminal device 3, and the authentication device 4 are configured to communicate with each other via a communication network NW. In this embodiment, the communication network NW is an IP (Internet Protocol) network, but there are no restrictions on the type of communication protocol, and furthermore, there are no restrictions on the type or size of the network.

[0029] In this embodiment, the information processing device 1 and authentication device 4 can be general-purpose server computers or personal computers. The participant terminal device 3 can be a smartphone, tablet, personal computer, or wearable device.

[0030] <1.2. Hardware Configuration of the Invention> Figure 2 is a hardware configuration diagram of the facial recognition system 0. As shown in Figure 2(a), the server 10 (information processing device 1, authentication device 4) comprises a processing unit 101, a storage unit 102, and a communication unit 103.

[0031] The processing unit 101 has one or more processors, such as a CPU, capable of executing instruction sets, and controls the entire operation of the server 10 by executing the face recognition program, OS, and other applications according to the present invention. The memory unit 102 includes a volatile memory such as RAM capable of storing instruction sets, an OS, and a non-volatile recording medium such as an HDD or SSD capable of storing the face recognition program according to the present invention. The communication unit 103 has a communication interface device with the communication network NW, and performs communication control with the communication network NW to input and output information.

[0032] As shown in Figure 2(b), the terminal device 9 (student terminal device 3) comprises a processing unit 91, a storage unit 92, a communication unit 93, an input unit 94, and an output unit 95.

[0033] The processing unit 91 has one or more processors, such as a CPU, capable of executing instruction sets, and controls the entire operation of the terminal device 9 by executing an information processing device user program, an OS, and other applications for using the information processing device 1. The memory unit 92 includes volatile memory such as RAM capable of storing instruction sets, and non-volatile recording media such as HDDs and SSDs capable of storing the OS and information processing device utilization programs. The communication unit 93 has a communication interface device for connecting to a network and performs communication control with the communication network NW to input and output information. The input unit 94 includes an input device capable of input processing, such as a keyboard or touch panel, and an imaging device capable of acquiring the participant's facial information and the distance between the participant and the terminal device. Here, the imaging device is configured to detect not only visible light but also various electromagnetic waves (infrared, ultraviolet, etc.) (for example, an infrared camera, depth sensor, LiDAR (Light Detection And Ranging), etc.). The output unit 95 has a display device capable of display processing, such as a display. The output unit 95 may also be an output unit located outside the terminal device 9.

[0034] <1.2. System Functional Configuration> Figure 3 is a functional block diagram of the facial recognition system 0 in this embodiment. As shown in Figure 3, the information processing device 1 comprises a registration unit 11, an acquisition unit 12, a tracking processing unit 13, an image adjustment unit 14, an authentication processing unit 15, a history management unit 16, a display processing unit 17, and an online learning database 2. This is a concrete implementation of the facial recognition program stored in the storage unit 102 by the processing unit 101.

[0035] In this embodiment, the system configuration is a so-called server-client type, where the client receives the processing results performed by the information processing device 1 (server) in response to a request from the student terminal device 3 (client). Alternatively, it may be a so-called standalone type, where the client terminal activates a facial recognition program. In this case, the student terminal device 3 may include some or all of the functional components (parts) of the information processing device 1. For example, the student terminal device 3 may include a registration unit 11, an acquisition unit 12, a tracking processing unit 13, an image adjustment unit 14, an authentication processing unit 15, a history management unit 16, and a display processing unit 17, and the information processing device 1 may function as a device for storing online learning videos.

[0036] Furthermore, the information processing device 1 may be composed of multiple computers that are capable of sending and receiving information via a communication network NW or another network, and the display processing unit 17, etc., of the information processing device 1 may be provided on a server separate from the information processing device 1. Also, some or all of the functional configuration of the information processing device 1 may be provided on the authentication device 4.

[0037] <1.2.1. Online Learning Database 2> Online learning database 2 stores student information, learning video information, and learning history information.

[0038] <1.2.1.1. Participant Information> Student information refers to information about students taking online courses. This information includes a unique student ID, student name, contact information, and login password.

[0039] <1.2.1.2. Learning Video Information> Learning video information refers to information about videos played during online learning. This learning video information includes a video ID that uniquely identifies the online learning video, the video content, and an administrator ID that uniquely identifies the administrator of the online learning.

[0040] <1.2.1.3. Course History Information> The history management unit 16 registers learning history information, which is information about the online learning history taken by each student. The learning history information includes a learning history ID that uniquely identifies the learning history, a student ID, a video ID, a learning status (not completed / completed), and the playback date and time.

[0041] <1.2.2. Registration Section 11> The registration unit 11 registers the authenticated facial information in the authentication database 5 of the authentication device 4. The registration unit 11 adjusts the specifications of the authenticated facial information to predetermined values ​​and registers it in the authentication database 5.

[0042] Here, the authentication facial information is registered in advance before the online learning session and is used for authentication. This registration includes a combination of the participant's facial image data and facial features (such as the arrangement and contour of facial parts like the face, nose, eyes, eyebrows, and mouth, and skin pattern). The facial image data is also adjusted to predetermined image specifications (size, aspect ratio, brightness, color, etc.). In addition to the combination of facial image data and facial features, or alternatively, 3D point cloud data, mesh data, depth data, etc., of the face may be used as authentication facial information.

[0043] The registration unit 11 also registers face information that failed face tracking into the online learning database 2. The registration unit 11 also registers multiple face image data that failed face tracking into the online learning database 2. The registration unit 11 may also register multiple face image data that failed face tracking into the authentication database 5.

[0044] <1.2.3. Acquisition part 12> The acquisition unit 12 acquires captured facial information. The acquisition unit 12 acquires facial image data as facial information of the participant during the online learning session displayed. In this embodiment, the acquisition unit 12 acquires facial image data as facial information, but in addition to or instead of facial image data, it may acquire 3D point cloud data, mesh data, depth data, etc. of the face.

[0045] <1.2.4. Tracking Processing Unit 13> The tracking processing unit 13 performs face tracking of the participant. The tracking processing unit 13 performs face tracking based on the acquired face information.

[0046] <1.2.5. Image Adjustment Section 14> The image adjustment unit 14 adjusts the face information. The image adjustment unit 14 adjusts the face information based on the face information obtained from the successful face tracking process.

[0047] Furthermore, the image adjustment unit 14 identifies the areas to be authenticated and the areas to be not authenticated in the facial information based on the facial contours of the participants included in the facial image data for which the facial tracking process was successful, and adjusts the areas to be not authenticated.

[0048] <1.2.6. Authentication Processing Unit 15> The authentication processing unit 15 performs the authentication process for the participant. The authentication processing unit 15 performs the authentication process based on the facial information obtained from successful facial tracking and the authenticated facial information.

[0049] <1.2.7. Display Processing Unit 17> The display processing unit 17 processes the online learning materials for display and displays the results on the student terminal device 3.

[0050] <2. Processing Flow> Next, with reference to Figures 4 and 5, a facial recognition method using the facial recognition system 0 of the present invention will be described. Figure 4 is a flowchart showing the process from when the information processing device 1 registers the authenticated facial information, when the student is authenticated during online learning, and when the student's learning history is updated.

[0051] <2.1. Registration of Authentication Face Information> In step S1 (hereinafter, "step SX" will simply be referred to as "SX"), the registration unit 11 registers the authenticated face information. In this embodiment, the registration unit 11 obtains the participant's face image data and the participant's participant ID from an external system. Then, by adjusting the image specifications of the face image data to predetermined values, the adjusted face image data is used as the face image data for the authenticated face information. This face image data is then input into a well-known image recognition model to identify face features, thereby registering the authenticated face information in the authentication database 5.

[0052] <2.2. Display processing for online learning> In S2, the display processing unit 17 processes the online learning to display. In this embodiment, the student terminal device 3 receives a request from the student to specify the online learning video they wish to take, and transmits the video ID of the video and the student's student ID to the information processing device 1. The display processing unit 17 processes the online learning video based on the video ID and displays it on the student terminal device 3. The history management unit 16 also creates a record of learning history information including the combination of the student ID and video ID, and registers the learning status of the learning history information record as "incomplete" in the online learning database 2.

[0053] <2.3. Acquisition of facial image data> In S3, the acquisition unit 12 acquires facial image data. In this embodiment, the acquisition unit 12 sends an instruction to the participant terminal device 3 to acquire facial image data at predetermined time intervals, and acquires the participant's facial image data during the course via the participant terminal device 3.

[0054] <2.4. Execution of Face Tracking Process> In S4, the tracking processing unit 13 performs face tracking. In this embodiment, the tracking processing unit 13 performs face tracking based on the face image data acquired in S3. Details of the face tracking process will be described later with reference to Figure 5.

[0055] <2.5. Adjusting facial image data> In S5, the image adjustment unit 14 adjusts the face image data. In this embodiment, the image adjustment unit 14 scales the face image data based on the face image data for which face tracking processing was successful in S4. Specifically, the image adjustment unit 14 adjusts the size of the face image data for which face tracking processing was successful to a predetermined size (for example, 300 x 300 pixels). The image adjustment unit 14 also identifies the face region of the participant from the captured face image data of the participant and adjusts the face image data by cropping the face region so as to leave a predetermined margin around the face region. Specifically, the image adjustment unit 14 crops the face region and adjusts the face image data using a rectangular boundary line that circumscribes the face region. Alternatively, the image adjustment unit 14 may crop the face region and adjust the face image data using a boundary line that follows the shape of the face region.

[0056] In addition, the image adjustment unit 14 adjusts the aspect ratio of the face image data from which the face tracking process has been successful to a predetermined ratio. For example, the image adjustment unit 14 may identify the face region of the participant from the captured face image data of the participant and adjust the aspect ratio of the face image data based on the proportion of the face region that the face region occupies within the face image data. These measures not only reduce the file size of the facial image data itself, but also reduce the amount of extraneous area in the facial image data where the face is not visible, according to the contours of the participant's face, thereby reducing the load on the authentication process described later.

[0057] Furthermore, the image adjustment unit 14 identifies the authentication target area and non-authentication target area included in the facial image data in which the face tracking process was successful in S4, and adjusts the authentication target area and non-authentication target area.

[0058] Specifically, the authentication target area is the participant's face, and the image adjustment unit 14 adjusts the brightness of the participant's face (for example, adjusting the pixel brightness to a range of 180 to 220). Furthermore, the non-authenticated area is the area of ​​the face image data other than the participant's face (such as the background area in the margin, the participant's hair area, and the trunk area including the limbs and torso below the neck), and the image adjustment unit 14 adjusts the brightness of the participant's non-authenticated area (for example, adjusting the pixel brightness to a range of 40 to 70). The image adjustment unit 14 also performs monochrome processing on the participant's non-authenticated area to adjust the face image data. The image adjustment unit 14 also performs image filtering on the participant's non-authenticated area to adjust the face image data. Here, image filtering processes such as blurring or mosaic effects are used to hide parts of the image. This makes the participant's face clearer, which improves the accuracy of the authentication process described later.

[0059] Furthermore, the image adjustment unit 14 adjusts the area to be authenticated and the visual characteristic values ​​of the area to be authenticated so that the visual characteristic ratio based on the visual characteristic values ​​of the area to be authenticated and the visual characteristic values ​​of the area not to be authenticated becomes a predetermined ratio. In this embodiment, the visual characteristic value is luminance, and the area to be authenticated and the luminance of the area to be authenticated are adjusted so that the luminance ratio (luminance contrast) based on the luminance of the area to be authenticated and the luminance of the area not to be authenticated becomes 3:1 to 5:1. Note that the visual characteristic value may also be a color value (RGB, CMKY, etc.). This makes the participant's face area clearer than the non-face area, thereby improving the accuracy of the authentication process described later.

[0060] In a preferred embodiment of this design, the image adjustment unit 14 performs adjustments on face image data in which the brightness of the authentication target area and the non-authentication target area has been adjusted. Specifically, the image adjustment unit 14 performs monochrome processing on the non-authentication target area of ​​face image data in which the brightness of the non-authentication target area is lower than that of the authentication target area, thereby adjusting the face image data. In addition, the image adjustment unit 14 performs image filtering processing on the non-authentication target area of ​​face image data in which the brightness of the non-authentication target area is higher, thereby adjusting the face image data. The brightness ratio (brightness contrast) between the brightness of the area to be authenticated and the brightness of the area not to be authenticated, at which both monochrome processing and image filtering can be applied most effectively, is between 3:1 and 5:1, which can further improve the accuracy of the authentication process described later.

[0061] <2.6. Execution of Authentication Process> In S6, the authentication processing unit 15 performs the student authentication process. In this embodiment, the authentication processing unit 15 performs the authentication process by comparing the facial feature quantities of the facial image data of the authentication facial information corresponding to the student ID obtained in S2 with the facial feature quantities of the facial image data adjusted in S5, based on the facial image data adjusted in S5, the authentication facial information registered in S1, and the student ID obtained in S2.

[0062] <2.7. Authentication Process Determination> In S7, the authentication unit 41 determines whether the authentication process was successful or not. If the authentication process fails (NO in S7), the display processing unit 17 displays the online learning stop screen (S8). Here, the stop screen may be a screen where the online learning video is stopped and a face recognition start button is displayed, or it may be a screen without a face recognition start button, prompting the student to wait until face recognition starts. When the online learning video is displayed, processing from S3 onwards resumes, and the online learning stop screen is displayed until it is determined that the authentication process was successful (YES in S7).

[0063] On the other hand, if the authentication process is successful (YES in S7), the display processing unit 17 determines whether there is remaining time in the online learning video specified in S2. If there is remaining time in the learning video (YES in S9), the process returns to S2, and the display processing unit 17 continues to display the online learning video. On the other hand, if there is no remaining time in the learning video (NO in S9), the process proceeds to S10.

[0064] <2.8. Registration of Course Attendance History> In S10, the history management unit 16 updates the course history information. In this embodiment, the history management unit 16 identifies the course history information record corresponding to the combination of the student ID obtained in S2 and the specified video ID, and updates the course status of the course history information record to completed.

[0065] <3. Details of the facial tracking process> The following describes the specific process involved in face tracking in S4 of Figure 4, with reference to Figure 5. In the following process, if face tracking fails more than a predetermined number of times, the face image data from the failed face tracking attempts is used to perform face recognition of the participant.

[0066] <3.1. Determining the success of face tracking processing> First, in S41, the tracking processing unit 13 determines whether the face tracking process was successful or not. If the face tracking process is successful (YES in S41), the process proceeds to S5 in Figure 4. On the other hand, if the face tracking process fails (NO in S41), the process proceeds to S42.

[0067] <3.2. Registration of facial image data> In S42, the registration unit 11 registers the face image data for which the face tracking process failed. In this embodiment, the registration unit 11 registers the face image data for which the face tracking process failed in S41 and the participant ID acquired in S2, linking them together.

[0068] <3.3. Checking the number of failures in face tracking processing> In S43, the tracking processing unit 13 determines whether the number of face tracking failures exceeds a predetermined allowable number. If the number of face tracking failures does not exceed the predetermined allowable number (YES in S43), the process returns to S3, and the acquisition unit 12 continues to acquire face image data of students taking online lessons. On the other hand, if the number of face tracking failures exceeds the predetermined allowable number (NO in S43), the process proceeds to S44.

[0069] <3.4. Identification of Facial Features> In S44, the authentication processing unit 15 identifies the facial features of the student based on multiple facial image data for which the facial tracking process failed. In this embodiment, the authentication processing unit 15 acquires multiple facial image data registered in S42 corresponding to the student ID acquired in S2, and inputs each facial image data into a well-known image recognition model to identify the pseudo-facial features of the student's facial parts captured in each facial image data. Then, the authentication processing unit 15 identifies the facial features of the student based on the multiple pseudo-facial features.

[0070] Specifically, the authentication processing unit 15 creates face position-related information based on multiple pseudo-face features and identifies the participant's face features based on the face position-related information. More specifically, the authentication processing unit 15 applies well-known 3D shape estimation techniques (Structure from Motion (SfM), 3D Morphable Model (3DMM)) to the pseudo-face features identified from each face image data to create 3D point cloud data based on the pseudo-face features as face position-related information, and generates a 3D model based on the 3D point cloud data. Then, the authentication processing unit 15 rotates the generated 3D model to face forward and identifies the participant's face features (such as the arrangement and contour of facial features like face, nose, eyes, eyebrows, and mouth) based on the image data of the front face.

[0071] In addition, the authentication processing unit 15 identifies the pseudo-face features (position coordinates of landmarks such as the nose and eyes) of the participant in each face image data from multiple face image data from which the face tracking process failed. The authentication processing unit 15 then applies a well-known coordinate transformation technique (such as Procrustes Analysis) to these pseudo-face features to create a reference coordinate system that unifies the coordinate systems of each face image data as face position-related information. The authentication processing unit 15 then applies a well-known geometric interpolation technique (such as linear interpolation, weighted average interpolation, principal component analysis) to the pseudo-face features of each face image data that have been transformed (translated, scaled, rotated, etc.) into the reference coordinate system to interpolate the pseudo-face features of each face image data, thereby identifying the participant's face features (the arrangement and contour of facial parts such as the face, nose, eyes, eyebrows, and mouth). In other words, a reference coordinate system is created from the position coordinates of each facial part in each facial image data. If there are missing landmarks in any of the facial image data within this reference coordinate system, the position coordinates of the missing parts are identified by using other facial image data that does not contain those missing parts, thereby determining the position coordinates of all parts of the participant's face.

[0072] The authentication processing unit 15 may perform the following operations in reverse order: the interpolation of pseudo-face features for each face image data and the creation of a reference coordinate system based on the pseudo-face features for each face image data. Alternatively, the face image data from which the pseudo-face features have been interpolated may be used to create a reference coordinate system, and then the positional coordinates of the entire face of the participant may be identified.

[0073] Furthermore, in this embodiment, the identification process refers to the process by which the authentication processing unit 15 identifies the facial features of the participant based on the facial features of each facial image data for which the facial tracking process failed. Alternatively, the authentication processing unit 15 may transmit an instruction to the authentication device 4 to identify the facial features of each facial image data for which the facial tracking process failed, as well as the facial features of the participant, and the authentication unit 41 may then identify the facial features of the participant.

[0074] <3.5. Comparison of Facial Feature Values> In S45, the authentication processing unit 15 performs authentication using the facial features based on the failed facial image data and the facial features of the authenticated facial information. If the facial features identified in S44 and the facial features of the authenticated facial information registered in S1 of Figure 4 match (YES in S45), the process returns to S3, and the acquisition unit 12 continues to acquire facial image data of the student taking the online lesson. On the other hand, if the facial features identified in S44 and the facial features of the authenticated facial information registered in S1 of Figure 4 do not match (NO in S45), the process proceeds to S8, and the display processing unit 17 displays the pause screen for the online learning video.

[0075] As described above, the execution of processes S1 to S10 prevents fraudulent activity by students and avoids unnecessary facial recognition errors. Furthermore, by appropriately adjusting the facial image data from successful facial tracking, facial recognition errors are reduced, minimizing interruptions to online learning due to facial recognition errors. This reduces stress for students.

[0076] In this embodiment, a combination of face image data and face features is registered as authentication face information, but only face image data may be registered. In this case, the authentication unit 41 may, upon receiving face information and a student ID from the authentication processing unit 15, identify the face features of the authentication face information corresponding to the student and perform authentication of the student.

[0077] Furthermore, the authentication process in this embodiment involves transmitting the face information obtained from successful face tracking, the student ID, and an authentication instruction to the authentication device 4, whereupon the authentication unit 41 identifies the face features of the face information and authenticates the student based on these face features and the face features of the authentication face information corresponding to the student ID among the authentication face information registered in the authentication database 5. Alternatively, the authentication processing unit 15 may identify the face features of the face information obtained from successful face tracking, and the authentication unit 41 may authenticate the student based on these face features and the face features of the authentication face information corresponding to the student ID among the authentication face information registered in the authentication database 5.

[0078] In this embodiment, the display process refers to the process in which the display processing unit 17 executes a process to generate information necessary for display, and transmits the generated information to the terminal device 90, causing the terminal device 90 to display the generated information. Alternatively, the display process may also refer to the process in which the display processing unit 17 transmits a processing command to the terminal device 90 to generate information necessary for display, causing the terminal device 90 to generate the information necessary for display and display the generated information. Furthermore, if the display processing unit 17 is provided in the terminal device 90 (in the case of a standalone type), the display process may refer to the process in which the display processing unit 17 executes a process to generate the necessary information, and transmits the generated information to the output unit 95 of the terminal device 90, causing the output unit 95 to display the generated information. [Explanation of Symbols]

[0079] 0: Facial recognition system 1: Information Processing Device 2: Distribution Database 3: Participant terminal device 4: Authentication device 5: Authentication Database 10: Server 101: Processing Unit 102: Storage section 103: Communications Department 9: Terminal device 91: Processing Unit 92: Storage section 93: Communications Department 94: Input section 95: Output section 11: Registration Department 12: Acquisition part 13: Tracking Processing Unit 14: Image adjustment section 15: Authentication Processing Unit 16: Display Processing Unit 41: Authentication Department NW: Communication Network

Claims

1. A facial recognition system for performing facial recognition on online learning participants, The aforementioned facial recognition system comprises a database, a history management unit, an acquisition unit, a tracking processing unit, and an authentication processing unit. The database stores authentication facial information for authenticating the student, and learning video information for the videos used for online learning. The acquisition unit acquires the facial information of the participant during the online learning session that is displayed. The history management unit links the participant, the video specified by the participant, and the participation status indicating the participant's participation history for the video, and registers them in the database. The tracking processing unit performs face tracking processing based on the face information and determines whether or not it has succeeded in obtaining face information that can be authenticated by at least the authentication processing unit. The authentication processing unit executes an authentication process based on the face information obtained from the successful face tracking process and the authenticated face information. Furthermore, the multiple face information from which the face tracking process failed is input into a predetermined image recognition model to identify the pseudo-face features of the participant's face for each of the multiple face information from which the face tracking process failed, and the authentication process is performed based on the face features identified based on the pseudo-face features and the face features of the authenticated face information. The history management unit updates the course status associated with the video based on the remaining time of the video when the authentication process is successful. Facial recognition system.

2. The tracking processing unit determines that if the face tracking process fails a predetermined number of times, it will stop the online learning. The authentication processing unit determines, if the authentication process is successful, that it will resume the suspended online learning session. The facial recognition system according to claim 1.

3. The aforementioned facial recognition system further includes an image adjustment unit, The image adjustment unit scales the face information based on the face information from which the face tracking process was successful. The authentication processing unit performs authentication processing based on the scaled face information and the authenticated face information. The facial recognition system according to claim 1.

4. The aforementioned facial recognition system further includes an image adjustment unit, The image adjustment unit identifies the authentication target area and the non-authentication target area included in the facial information based on the facial contour of the participant included in the facial information. Adjust the aforementioned unauthenticated area The facial recognition system according to claim 1.

5. The image adjustment unit adjusts the area to be authenticated and the visual characteristic values ​​of the area to be authenticated so that the visual characteristic ratio, based on the visual characteristic values ​​of the area to be authenticated and the visual characteristic values ​​of the area not to be authenticated, becomes a predetermined ratio. The facial recognition system according to claim 4.

6. A facial recognition program that performs facial recognition on online learning participants, The computer will function as a database, history management unit, acquisition unit, tracking processing unit, and authentication processing unit. The database stores authentication facial information for authenticating the student, and learning video information for the videos used for online learning. The acquisition unit acquires the facial information of the participant during the online learning session that is displayed. The history management unit links the participant, the video specified by the participant, and the participation status indicating the video's participation history, and registers them in the database. The tracking processing unit performs face tracking processing based on the face information and determines whether or not it has succeeded in obtaining face information that can be authenticated by at least the authentication processing unit. The authentication processing unit executes an authentication process based on the face information obtained from the successful face tracking process and the authenticated face information. Furthermore, the multiple face information from which the face tracking process failed is input into a predetermined image recognition model to identify the pseudo-face features of the participant's face for each of the multiple face information from which the face tracking process failed, and the authentication process is performed based on the face features identified based on the pseudo-face features and the face features of the authenticated face information. The history management unit updates the course status associated with the video based on the remaining time of the video when the authentication process is successful. Facial recognition program.

7. A facial recognition program that performs facial recognition on online learning participants, Computers A process for storing authentication facial information for authenticating the aforementioned student, and learning video information for the videos used for online learning, A process for obtaining the facial information of the participant during the online learning session that is displayed, A process to link the aforementioned student, the video specified by the student, and the student status indicating the student's learning history for the video, and register them in a database. A process to perform face tracking based on the aforementioned face information and determine whether to obtain at least verifiable face information, A process that performs an authentication process based on the face information obtained from the successful face tracking process and the authenticated face information, Furthermore, the process involves inputting multiple face information for which the face tracking process failed into a predetermined image recognition model, identifying pseudo-face features of the participant's face area for each of the multiple face information for which the face tracking process failed, and executing an authentication process based on the face features identified based on the pseudo-face features and the face features of the authentication face information. A process to update the learning status associated with the video based on the remaining time of the video when the authentication process is successful, A facial recognition method that performs this task.

Images