Facial recognition system, facial recognition program, and facial recognition method

Abstract

An object of the present invention is to provide a technology that enables appropriate authentication processing for online learning. [Solution] In order to solve the above problems, the present invention provides a face authentication system for performing face authentication on students taking online learning courses, the face authentication system comprising a database, an acquisition unit, a tracking processing unit, and an authentication processing unit, the database stores authentication face information for authenticating the students, the acquisition unit acquires face information of the students while they are taking the displayed online learning course, the tracking processing unit performs face tracking processing based on the face information, and the authentication processing unit performs authentication processing based on the face information for which the face tracking processing was successful and the authentication face information.

Description

[Technical Field]

[0001] The present invention relates to a face recognition system, a face recognition program, and a face recognition method. [Background technology]

[0002] In recent years, e-learning has rapidly become popular due to the digitalization of learning environments and educational settings. However, the spread of such learning systems has led to a problem of an increase in the number of students who engage in fraudulent behavior, and facial recognition to prevent such fraudulent behavior has become increasingly important. An example of such facial recognition technology is proposed in, for example, Patent Document 1.

[0003] For example, Patent Document 1 describes that even if a test taker is located in a remote location, the test taker's identity can be confirmed by processing the image for personal identification, which is pre-registered as the test taker's facial image, so that it is compatible with identity verification processing that confirms that the test taker is the actual test taker during the test, and the test can be conducted while ensuring that the test taker is the actual test taker. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-212002 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Patent Document 1, even if an e-learning examinee changes their posture naturally while taking the exam, it may not be possible to verify their identity, which can result in the examinee being judged to be engaging in cheating even when they are not, which is very inconvenient for the examinee.

[0006] In view of the above problems, an object of the present invention is to provide a technology that can appropriately perform authentication processing for online learning. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention provides a face authentication system for performing face authentication on students taking online learning courses, the face authentication system comprising a database, an acquisition unit, a tracking processing unit, and an authentication processing unit, the database stores authentication face information for authenticating the students, the acquisition unit acquires face information of the students while they are taking the displayed online learning course, the tracking processing unit performs face tracking processing based on the face information, and the authentication processing unit performs authentication processing based on the face information for which the face tracking processing was successful and the authentication face information.

[0008] In addition, in order to solve the above-mentioned problems, the present invention provides a face recognition program for performing face recognition of students taking online learning courses, which causes a computer to function as a database, an acquisition unit, a tracking processing unit, and an authentication processing unit, wherein the database stores authentication face information for authenticating the students, the acquisition unit acquires face information of the students while they are taking the displayed online learning course, the tracking processing unit performs face tracking processing based on the face information, and the authentication processing unit performs authentication processing based on the face information for which the face tracking processing was successful and the authentication face information.

[0009] In addition, in order to solve the above-mentioned problems, the present invention provides a face authentication method for performing face authentication of students taking online learning courses, in which a computer performs the following processes: storing authentication face information for authenticating the students; acquiring face information of the students while they are taking the displayed online learning course; performing face tracking processing based on the face information; and performing authentication processing based on the face information for which the face tracking processing was successful and the authentication face information.

[0010] This configuration significantly reduces authentication process failures, thereby preventing fraud and enabling stress-free online learning.

[0011] In a preferred embodiment of the present invention, the tracking processing unit determines to stop the online learning if the face tracking process fails a predetermined number of times, and the authentication processing unit determines to resume the stopped online learning if the authentication process is successful.

[0012] This configuration can prevent unauthorized online learning by making unnecessary facial movements to prevent the authentication process from being executed.

[0013] In a preferred embodiment of the present invention, the face authentication system further includes an image adjustment unit that scales the face information based on the face information for which the face tracking process was successful, and the authentication processing unit performs authentication processing based on the scaled face information and the authentication face information.

[0014] This configuration can reduce the processing load of the authentication process. For example, by adjusting the size and aspect ratio of the facial image data to reduce unnecessary parts of the facial image data, the volume of the facial image data can be reduced, thereby reducing the processing load of the authentication process.

[0015] In a preferred embodiment of the present invention, the facial recognition system further includes an image adjustment unit that identifies areas to be recognized and areas not to be recognized that are included in the facial information based on the facial contour of the student included in the facial information, and adjusts the areas not to be recognized.

[0016] With this configuration, it is possible to improve the accuracy of the authentication process using the authentication target area.

[0017] In a preferred embodiment of the present invention, the image adjustment unit adjusts the visual characteristic values ​​of the authentication target area and the authentication target area so that a visual characteristic ratio based on the visual characteristic values ​​of the authentication target area and the visual characteristic values ​​of the non-authentication target area becomes a predetermined ratio.

[0018] With this configuration, it is possible to improve the accuracy of the authentication process for the authentication target area.

[0019] In a preferred embodiment of the present invention, the authentication processing unit executes the authentication process based on the facial feature amounts of the plurality of pieces of face information for which the face tracking process has failed and the facial feature amounts of the authentication face information.

[0020] This configuration reduces the frequency of online learning interruptions due to face tracking failures, making it possible to reuse multiple pieces of face information that have failed in face tracking processing, further reducing stress for students taking the course. [Effects of the Invention]

[0021] The present invention has an effect of providing a technique that enables appropriate authentication processing for online learning. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a block diagram showing a configuration of a system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a hardware configuration diagram of a system according to an embodiment of the present invention. [Figure 3] 1 is a functional block diagram of an apparatus according to an embodiment of the present invention. [Figure 4] 1 is a processing flowchart of a system according to an embodiment of the present invention. [Figure 5] 10 is a detailed flowchart of a face tracking process according to an embodiment of the present invention. [Figure 6] 1 is a processing flowchart of a conventional face authentication system for online learning. DETAILED DESCRIPTION OF THE INVENTION

[0023] The present invention will now be described more fully with reference to the accompanying drawings, in which preferred embodiments are shown, but which may be embodied in many different forms and are not limited to the embodiments set forth herein.

[0024] For example, although the configuration, operation, etc. of a face authentication system are described in this embodiment, methods, devices, computer programs, etc. with similar configurations can also achieve similar effects. Furthermore, the program may be stored on a recording medium or provided as a downloadable program from an external server. Furthermore, by using this recording medium, for example, the program can be installed on a computer, thereby configuring a face authentication device and a face authentication system. Here, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.

[0025] <1. Overview of the present invention> This invention relates to a facial recognition system that performs facial recognition for students in online learning. Conventional facial recognition systems often use a method (see Figure 6) in which facial images are captured at regular intervals, designed to prevent fraud, and authentication is performed each time. However, this method has the problem that students may move their faces away from the web camera when referring to textbooks or notes, or when their posture changes naturally while they are thinking. This results in frequent authentication errors, which not only interrupts the learning process but also increases the number of re-authentication processes. This problem can also be stressful for students and potentially lead to reduced learning efficiency.

[0026] Another problem is that while conventional facial recognition systems often suspect cheating when a certain number of consecutive authentication errors occur, this method makes it difficult to clearly distinguish between temporary authentication errors caused by legitimate reasons, such as a natural change in posture while thinking, and cheating, such as intentionally leaving the seat and not listening to the lecture. As a result, there is a risk of false positives being detected as cheating even when there is no cheating, or of cheating being overlooked.

[0027] To address this issue, the present invention employs a two-stage authentication process: face tracking and authentication. First, in the face tracking process, facial information of the student taking the course is acquired via the student's terminal device, who is taking online courses on a browser, and a determination is made as to whether the facial information has been captured accurately. If the facial information has been captured accurately, the authentication process then performs personal authentication using the facial information to determine whether the student is taking the course without engaging in any fraudulent activity.

[0028] <1.1. System Configuration> Fig. 1 is a block diagram showing the system configuration of the present invention. As shown in Fig. 1, the face authentication system 0 includes an information processing device 1, a student terminal device 3, and an authentication device 4. In this embodiment, the information processing device 1, the student terminal device 3, and the authentication device 4 are configured to be able to communicate with each other via a communication network NW. The communication network NW in this embodiment is an IP (Internet Protocol) network, but there is no restriction on the type of communication protocol, and there is also no restriction on the type or scale of the network.

[0029] In this embodiment, a general-purpose server computer or a personal computer can be used as the information processing device 1 and the authentication device 4. Furthermore, a smartphone, a tablet terminal, a personal computer, a wearable device, or the like can be used as the student terminal device 3.

[0030] <1.2. Hardware configuration of the present invention> 2 is a hardware configuration diagram of the face authentication system 0. As shown in FIG. 2(a), the server 10 (information processing device 1, authentication device 4) includes a processing unit 101, a storage unit 102, and a communication unit 103.

[0031] The processing unit 101 has one or more processors such as a CPU capable of executing an instruction set, and controls the overall operation and processing of the server 10 by executing the face recognition program according to the present invention, an OS, and other applications. The storage unit 102 has a volatile memory such as a RAM capable of storing an instruction set, an OS, and a non-volatile recording medium such as an HDD or SSD capable of recording the face recognition program according to the present invention. The communication unit 103 has a communication interface device with the communication network NW, and controls communication with the communication network NW to input and output information.

[0032] As shown in FIG. 2(b), the terminal device 9 (student terminal device 3) includes a processing unit 91, a storage unit 92, a communication unit 93, an input unit 94, and an output unit 95.

[0033] The processing unit 91 has one or more processors, such as a CPU, that can execute an instruction set, and controls the overall operation and processing of the terminal device 9 by executing an information processing device usage program, an OS, and other applications for using the information processing device 1. The storage unit 92 has a volatile memory such as a RAM capable of storing an instruction set, and a non-volatile recording medium such as an HDD or SSD capable of recording an OS, an information processing device utilization program, and the like. The communication unit 93 has a communication interface device for connecting to a network, and controls communication with the communication network NW to input and output information. The input unit 94 has an input device capable of input processing such as a keyboard or a touch panel, and an imaging device capable of acquiring face information of the students, the distance between the students and the terminal device, etc. Here, the imaging device is configured to be able to detect not only visible light but also various electromagnetic waves (infrared rays, ultraviolet rays, etc.) (for example, an infrared camera, a depth sensor, LiDAR (Light Detection and Ranging), etc.). The output unit 95 has a display device capable of display processing such as a display etc. The output unit 95 may be an output unit external to the terminal device 9.

[0034] <1.2. System Functional Configuration> Fig. 3 is a functional block diagram of a face recognition system 0 according to this embodiment. As shown in Fig. 3, the information processing device 1 includes a registration unit 11, an acquisition unit 12, a tracking processing unit 13, an image adjustment unit 14, an authentication processing unit 15, a history management unit 16, a display processing unit 17, and an online learning database 2. This is a specific implementation of a face recognition program stored in a storage unit 102 by a processing unit 101.

[0035] The system configuration in this embodiment is a so-called server-client type in which the client receives the processing results performed by the information processing device 1 (server) in response to a request from the student terminal device 3 (client). Alternatively, a so-called standalone type in which a face recognition program is started on the client terminal may also be used. In this case, the student terminal device 3 may include some or all of the functional components (units) included in the information processing device 1. For example, the student terminal device 3 may include a registration unit 11, an acquisition unit 12, a tracking processing unit 13, an image adjustment unit 14, an authentication processing unit 15, a history management unit 16, and a display processing unit 17, and the information processing device 1 may function as a device for storing online learning videos.

[0036] Furthermore, the information processing device 1 may be configured with a plurality of computers that are capable of transmitting and receiving information via a communication network NW or another network, and the display processing unit 17 and the like of the information processing device 1 may be provided in a server separate from the information processing device 1. Furthermore, some or all of the functional configuration of the information processing device 1 may be provided in the authentication device 4.

[0037] <1.2.1. Online Learning Database 2> The online learning database 2 stores student information, learning video information, and learning history information.

[0038] <1.2.1.1. Participant Information> The student information is information about students taking online courses, including a student ID that uniquely identifies the student, the student's name, contact information, and login password.

[0039] <1.2.1.2. Learning video information> The learning video information is information about the video played during online learning. The learning video information includes a video ID that uniquely identifies the online learning video, the video content, and an administrator ID that uniquely identifies the administrator of the online learning.

[0040] <1.2.1.3. Course history information> The history management unit 16 registers attendance history information, which is information about the history of online learning attended by each student. The attendance history information includes an attendance history ID that uniquely identifies the attendance history, a student ID, a video ID, attendance status (incomplete / completed), and playback date and time.

[0041] <1.2.2. Registration Section 11> The registration unit 11 registers the authentication face information in the authentication database 5 of the authentication device 4. The registration unit 11 adjusts the standard of the authentication face information to a predetermined value and registers it in the authentication database 5.

[0042] Here, the authentication face information is a combination of the student's facial image data and facial features (the arrangement and contours of facial parts such as the face, nose, eyes, eyebrows, and mouth, skin pattern, etc.) that is registered in advance before taking online learning and used for authentication. The facial image data is adjusted to predetermined image standards (size, aspect ratio, brightness, color, etc.). Note that the authentication face information may include 3D point cloud data, mesh data, depth data, etc. of the face in addition to or instead of the combination of facial image data and facial features.

[0043] Furthermore, the registration unit 11 registers face information for which the face tracking process has failed in the online learning database 2. The registration unit 11 registers a plurality of pieces of face image data for which the face tracking process has failed in the online learning database 2. The registration unit 11 may also register a plurality of pieces of face image data for which the face tracking process has failed in the authentication database 5.

[0044] <1.2.3. Acquisition part 12> The acquisition unit 12 acquires captured facial information. The acquisition unit 12 acquires facial image data as facial information of a student taking a displayed online learning course. In this embodiment, the acquisition unit 12 acquires facial image data as facial information, but may acquire 3D point cloud data, mesh data, depth data, etc. of the face in addition to or instead of the facial image data.

[0045] <1.2.4. Tracking Processing Unit 13> The tracking processing unit 13 executes face tracking processing of the student based on the acquired face information.

[0046] <1.2.5. Image Adjustment Unit 14> The image adjuster 14 adjusts the face information based on the face information for which the face tracking process has been successful.

[0047] Furthermore, the image adjustment unit 14 identifies the authentication target area and the non-authentication target area included in the face information based on the facial contour of the student included in the face image data for which face tracking processing has been successful, and adjusts the non-authentication target area.

[0048] <1.2.6. Authentication processing unit 15> The authentication processing unit 15 performs authentication processing for the student based on the face information for which the face tracking processing has been successful and the authenticated face information.

[0049] <1.2.7. Display processing unit 17> The display processing unit 17 performs display processing of the online learning, and causes the student terminal device 3 to display the display processing results.

[0050] <2. Processing flow> Next, a face authentication method using the face authentication system 0 of the present invention will be described with reference to Figures 4 and 5. Figure 4 is a flowchart showing the process in which the information processing device 1 registers the authentication face information, authenticates the student while he or she is taking an online class, and updates the student's course history.

[0051] <2.1. Registering facial information for authentication> In step S1 (hereinafter, "step SX" will be simply referred to as "SX"), the registration unit 11 registers authentication face information. In this embodiment, the registration unit 11 acquires face image data of the student and the student ID of the student from an external system. Then, the image format of the face image data is adjusted to a predetermined value, and the adjusted face image data is used as face image data of authentication face information. Furthermore, the face image data is input into a well-known image recognition model to identify facial features, and the authentication face information is registered in the authentication database 5.

[0052] <2.2. Display processing for online learning> In S2, the display processing unit 17 processes the display of the online learning. In this embodiment, the student terminal device 3 accepts a designation of the online learning video that the student wishes to take, and transmits the video ID of the video and the student ID of the student to the information processing device 1. The display processing unit 17 processes the display of the online learning video based on the video ID, and displays it on the student terminal device 3. The history management unit 16 also creates a record of attendance history information including the combination of the student ID and video ID, and registers the attendance status of the record of attendance history information in the online learning database 2 as incomplete.

[0053] <2.3. Acquiring facial image data> In S3, the acquisition unit 12 acquires face image data. In this embodiment, the acquisition unit 12 transmits an instruction to acquire face image data to the student terminal device 3 at predetermined time intervals, and acquires face image data of the student during the course via the student terminal device 3.

[0054] <2.4. Execution of face tracking processing> In S4, the tracking processing unit 13 executes face tracking processing. In this embodiment, the tracking processing unit 13 executes face tracking processing based on the face image data acquired in S3. Details of the face tracking processing will be described later with reference to FIG.

[0055] <2.5. Adjusting facial image data> In S5, the image adjustment unit 14 adjusts the face image data. In this embodiment, the image adjustment unit 14 scales the face image data based on the face image data for which the face tracking process was successful in S4. Specifically, the image adjustment unit 14 adjusts the size of the face image data for which the face tracking process was successful to a predetermined size (e.g., 300 x 300 pixels). Furthermore, the image adjustment unit 14 identifies the face area of ​​the student from the captured face image data of the student, and adjusts the face image data by cutting out the face area so as to leave a predetermined margin in the face area. Specifically, the image adjustment unit 14 adjusts the face image data by cutting out the face area using a rectangular boundary line circumscribing the face area. Note that the image adjustment unit 14 may also adjust the face image data by cutting out the face area using a boundary line shaped to fit the face area.

[0056] In addition, the image adjustment unit 14 adjusts the aspect ratio of the face image data for which the face tracking process has been successful to a predetermined ratio. For example, the image adjustment unit 14 may identify the face area of ​​the student from the captured face image data of the student, and adjust the aspect ratio of the face image data based on the area ratio that the face area occupies in the face image data. This not only reduces the file size of the facial image data itself, but also reduces the amount of unnecessary area in the facial image data that does not show the face, depending on the facial contours of the student, thereby reducing the load on the authentication process described below.

[0057] In addition, the image adjustment unit 14 identifies the authentication target area and non-authentication target area contained in the face information based on the facial contour of the student contained in the face image data for which the face tracking process was successful in S4, and adjusts the authentication target area and non-authentication target area.

[0058] Specifically, the authentication target area is the face area of ​​the student, and the image adjustment unit 14 adjusts the brightness of the face area of ​​the student (for example, adjusts the pixel brightness to a range of 180 to 220). The non-authentication target area is an area of ​​the face image data other than the face area of ​​the student (such as the background area in the margin, the student's hair area, and the trunk area including the limbs and torso below the neck), and the image adjustment unit 14 adjusts the brightness of the non-authentication target area of ​​the student (for example, adjusts the pixel brightness to a range of 40 to 70). The image adjustment unit 14 also performs monochrome processing on the non-authentication target area of ​​the student to adjust the face image data. The image adjustment unit 14 also performs image filtering on the non-authentication target area of ​​the student to adjust the face image data. Here, the image filtering process uses processes for hiding part of the image, such as blurring and mosaic processing. This makes the facial area of ​​the student clearer, thereby improving the accuracy of the authentication process described below.

[0059] Furthermore, the image adjustment unit 14 adjusts the visual characteristic values ​​of the authentication target area and the authentication target area so that the visual characteristic ratio based on the visual characteristic values ​​of the authentication target area and the visual characteristic values ​​of the non-authentication target area becomes a predetermined ratio. In this embodiment, the visual characteristic value is luminance, and the luminance of the authentication target area and the authentication target area is adjusted so that the luminance ratio (luminance contrast) based on the luminance of the authentication target area and the luminance of the non-authentication target area becomes 3:1 to 5:1. The visual characteristic value may be a color value (RGB, CMKY, etc.). This makes the facial area of ​​the student clearer than the non-facial area, thereby improving the accuracy of the authentication process described below.

[0060] In a preferred embodiment of this embodiment, the image adjustment unit 14 adjusts the facial image data in which the luminance of the authentication target area and the non-authentication target area has been adjusted. Specifically, the image adjustment unit 14 adjusts the facial image data by performing monochrome processing on the non-authentication target area of ​​the facial image data in which the luminance of the non-authentication target area has been lowered compared to the luminance of the authentication target area. The image adjustment unit 14 also adjusts the facial image data by performing image filtering processing on the non-authentication target area of ​​the facial image data in which the luminance of the non-authentication target area has been increased. The brightness ratio (brightness contrast) between the brightness of the area to be authenticated and the brightness of the area not to be authenticated, to which both monochrome processing and image filtering processing can be most effectively applied, is 3:1 to 5:1, which can further improve the accuracy of the authentication processing described below.

[0061] <2.6. Execution of authentication process> In S6, the authentication processing unit 15 performs authentication processing for the student. In this embodiment, the authentication processing unit 15 performs authentication processing by comparing the facial feature amounts of the facial image data of the authentication face information corresponding to the student ID acquired in S2 with the facial feature amounts of the facial image data adjusted in S5, based on the facial image data adjusted in S5, the authentication face information registered in S1, and the student ID acquired in S2.

[0062] <2.7. Authentication process determination> In S7, the authentication unit 41 determines whether the authentication process was successful. If the authentication process failed (NO in S7), the display processing unit 17 performs processing to display an online learning stop screen (S8). Here, the stop screen may be a screen in which the online learning video is stopped and a button to start facial authentication is displayed, or a screen without a button to start facial authentication and prompting the student to wait until facial authentication is started. Then, once the online learning video has been displayed, processing from S3 onwards is resumed, and the online learning stop screen is displayed until it is determined that the authentication process was successful (YES in S7).

[0063] On the other hand, if the authentication process is successful (YES in S7), the display processing unit 17 determines whether there is any time remaining in the course time for the online learning video specified in S2. If there is time remaining in the course time (YES in S9), the process returns to S2, and the display processing unit 17 continues displaying the online learning video. On the other hand, if there is no time remaining in the course time (NO in S9), the process proceeds to S10.

[0064] <2.8. Registering course history> In S10, the history management unit 16 updates the attendance history information. In this embodiment, the history management unit 16 identifies a record of attendance history information corresponding to the combination of the student ID acquired in S2 and the specified video ID, and updates the attendance status of the record of attendance history information to completed.

[0065] <3. Details of face tracking processing> Specific processing related to the face tracking processing in S4 of Fig. 4 will be described below with reference to Fig. 5. In the following processing, if the face tracking processing fails a predetermined number of times or more, face authentication of the student is performed by using the face image data of the failed face tracking processing.

[0066] <3.1. Determining success of face tracking processing> First, in S41, the tracking processing unit 13 determines whether the face tracking process has been successful. If the face tracking process has been successful (YES in S41), the process proceeds to S5 in Fig. 4. On the other hand, if the face tracking process has failed (NO in S41), the process proceeds to S42.

[0067] <3.2. Registering facial image data> In S42, the registration unit 11 registers the face image data for which the face tracking process has failed. In this embodiment, the registration unit 11 associates and registers the face image data for which the face tracking process has failed in S41 with the student ID acquired in S2.

[0068] <3.3. Checking the number of failures in face tracking processing> In S43, the tracking processing unit 13 determines whether the number of failures in the face tracking process exceeds a predetermined allowable number. If the number of failures in the face tracking process does not exceed the predetermined allowable number (YES in S43), the process returns to S3, and the acquisition unit 12 continues to acquire face image data of students taking online courses. On the other hand, if the number of failures in the face tracking process exceeds the predetermined allowable number (NO in S43), the process proceeds to S44.

[0069] <3.4. Identifying facial features> In S44, the authentication processing unit 15 performs processing to identify facial features of the student based on the multiple pieces of facial image data for which the face tracking processing failed. In this embodiment, the authentication processing unit 15 acquires the multiple pieces of facial image data registered in S42 corresponding to the student ID acquired in S2, and inputs each piece of facial image data into a well-known image recognition model to identify provisional facial features of the student's facial parts captured in each piece of facial image data. Then, the authentication processing unit 15 performs processing to identify facial features of the student based on the multiple provisional facial features.

[0070] Specifically, the authentication processing unit 15 creates face position-related information based on the multiple temporary facial features, and identifies the facial features of the student based on the face position-related information. More specifically, the authentication processing unit 15 applies well-known 3D shape estimation techniques (Structure from Motion (SfM) and 3D Morphable Model (3DMM)) to the temporary facial features identified from each piece of facial image data to create 3D point cloud data based on the temporary facial features as face position-related information, and generates a 3D model based on the 3D point cloud data. Then, the authentication processing unit 15 rotates the generated 3D model to face forward, and identifies the facial features of the student (the arrangement and contours of facial parts such as the face, nose, eyes, eyebrows, and mouth) based on the image data of the frontal face.

[0071] Additionally, the authentication processing unit 15 identifies the provisional facial features (position coordinates of landmarks such as the nose and eyes) of the student in each piece of facial image data from multiple pieces of facial image data for which face tracking processing failed.The authentication processing unit 15 then applies a well-known coordinate transformation technique (such as Procrustes Analysis) to the provisional facial features to create a reference coordinate system that unifies the coordinate systems of each piece of facial image data as face position-related information.The authentication processing unit 15 then applies a well-known geometric interpolation technique (such as linear interpolation, weighted average interpolation, or principal component analysis) to the provisional facial features of each piece of facial image data that has been transformed (translation, scaling, rotation, etc.) into the reference coordinate system to complement the provisional facial features of each piece of facial image data, thereby identifying the facial features of the student (the arrangement and contours of facial parts such as the face, nose, eyes, eyebrows, and mouth, etc.). In other words, a reference coordinate system is created from the position coordinates of each facial part of each piece of facial image data, and if there is a missing landmark in each piece of facial image data within the reference coordinate system, the position coordinate of the missing part is identified by using other facial image data that does not contain the missing part, and the position coordinates of the entire facial part of the student are identified.

[0072] The authentication processing unit 15 may reverse the order of the process of complementing the provisional facial feature amounts of each piece of face image data and the process of creating a reference coordinate system based on the provisional facial feature amounts of each piece of face image data.The reference coordinate system may be created using the face image data that has been subjected to the process of complementing the provisional facial feature amounts, and the position coordinates of all parts of the student's face may be identified.

[0073] The identification process in this embodiment refers to a process in which the authentication processing unit 15 identifies the facial feature amounts of the student based on the facial feature amounts of each piece of face image data for which face tracking processing has failed. Alternatively, the authentication processing unit 15 may transmit an instruction to the authentication device 4 to identify the facial feature amounts of each piece of face image data for which face tracking processing has failed and the facial feature amounts of the student, so that the authentication unit 41 identifies the facial feature amounts of the student.

[0074] <3.5. Comparison of facial features> In S45, the authentication processing unit 15 performs authentication processing using the facial feature amount based on the failed facial image data and the facial feature amount of the authentication facial information. If the facial feature amount identified in S44 matches the facial feature amount of the authentication facial information registered in S1 of Fig. 4 (YES in S45), the process returns to S3, and the acquisition unit 12 continues to acquire facial image data of the student taking the online learning course. On the other hand, if the facial feature amount identified in S44 does not match the facial feature amount of the authentication facial information registered in S1 of Fig. 4 (NO in S45), the process proceeds to S8, and the display processing unit 17 displays a paused screen of the online learning video.

[0075] As described above, by executing the processes of S1 to S10, it is possible to prevent students from committing fraud and to prevent unnecessary facial recognition errors. Furthermore, by appropriately adjusting the facial image data for which the face tracking process was successful, facial recognition errors can be reduced, and the number of interruptions to online learning due to facial recognition errors can be reduced. This reduces stress for students.

[0076] In this embodiment, a combination of facial image data and facial features is registered as the authentication face information, but only facial image data may be registered. In this case, upon receiving the facial information and the student ID from the authentication processing unit 15, the authentication unit 41 may identify the facial features of the authentication face information corresponding to the student and authenticate the student.

[0077] Furthermore, the authentication process in this embodiment refers to sending the face information for which the face tracking process has been successful, the student ID, and an authentication instruction to the authentication device 4, so that the authentication unit 41 identifies the facial features of the face information and authenticates the student based on the facial features and the facial features of the authentication face information corresponding to the student ID among the authentication face information registered in the authentication database 5. On the other hand, the authentication processing unit 15 may identify the facial features of the face information for which the face tracking process has been successful, and the authentication unit 41 may authenticate the student based on the facial features and the facial features of the authentication face information corresponding to the student ID among the authentication face information registered in the authentication database 5.

[0078] Furthermore, in this embodiment, the display processing refers to a process in which the display processing unit 17 executes a process of generating information necessary for display, and transmits the generated information to the terminal device 90, thereby causing the terminal device 90 to display the generated information. On the other hand, the display processing may also be a process in which the display processing unit 17 transmits a processing command to the terminal device 90 to generate information necessary for display, thereby causing the terminal device 90 to generate information necessary for display, and display the generated information. Furthermore, when the display processing unit 17 is provided in the terminal device 90 (in the case of a stand-alone type), the display processing may also be a process in which the display processing unit 17 executes a process of generating necessary information, and transmits the generated information to the output unit 95 of the terminal device 90, thereby causing the output unit 95 to display the generated information. [Explanation of symbols]

[0079] 0:Facial recognition system 1: Information processing equipment 2: Distribution database 3: Participant terminal device 4: Authentication device 5: Authentication database 10: Server 101: Processing section 102: Storage section 103: Communications Department 9: Terminal device 91: Processing section 92: Storage section 93: Communications Department 94: Input section 95: Output section 11: Registration section 12: Acquisition part 13: Tracking processing section 14: Image adjustment section 15: Authentication processing section 16: Display processing section 41: Authentication section NW: Communication network

Claims

1. A face recognition system for performing face recognition of online learning students, The face authentication system includes a database, an acquisition unit, a tracking processing unit, and an authentication processing unit; The database stores authentication face information for authenticating the students, the acquiring unit acquires face information of the student while attending the displayed online learning session; the tracking processing unit executes a face tracking process based on the face information, and determines whether or not face information that can be authenticated by at least the authentication processing unit has been acquired; The authentication processing unit executes authentication a plurality of times at regular intervals during the online learning, and if the acquisition fails, the authentication processing unit does not execute authentication at the timing, and after the authentication processing unit successfully acquires face information that can be authenticated, executes authentication processing based on the acquired face information and the authentication face information. Facial recognition system.

2. the tracking processing unit determines to stop the online learning when the face tracking process has failed a predetermined number of times; the authentication processing unit determines to resume the stopped online learning if the authentication processing is successful; The face authentication system according to claim 1 .

3. The face authentication system further includes an image adjustment unit; the image adjustment unit scales the face information based on the face information for which the face tracking process has been successful; The authentication processing unit performs authentication processing based on the scaled face information and the authentication face information. The face authentication system according to claim 1 .

4. The face authentication system further includes an image adjustment unit; the image adjustment unit identifies an authentication target area and a non-authentication target area included in the face information based on a facial contour of the student included in the face information; Adjust the non-authentication target area The face authentication system according to claim 1 .

5. The image adjustment unit adjusts the visual characteristic values ​​of the authentication target area and the non-authentication target area so that a visual characteristic ratio based on the visual characteristic values ​​of the authentication target area and the visual characteristic values ​​of the non-authentication target area becomes a predetermined ratio. The face authentication system according to claim 4 .

6. The authentication processing unit executes authentication processing based on facial feature amounts of the plurality of pieces of face information for which the face tracking processing has failed and facial feature amounts of the authentication face information. The face authentication system according to claim 1 .

7. A facial recognition program for performing facial recognition of online learning students, A computer is configured to function as a database, an acquisition unit, a tracking processing unit, and an authentication processing unit; The database stores authentication face information for authenticating the students, the acquiring unit acquires face information of the student while attending the displayed online learning course; the tracking processing unit executes a face tracking process based on the face information, and at least the authentication processing unit determines whether or not the face information that can be authenticated has been acquired; The authentication processing unit executes authentication a plurality of times at regular intervals in the online learning, and if the acquisition fails, the authentication processing unit does not execute authentication at the timing, and after the authentication processing unit successfully acquires authenticable face information, executes authentication processing based on the acquired face information and the authentication face information. Facial recognition program.

8. A face recognition method for performing face recognition of online learning students, The computer A process of storing authentication face information for authenticating the student; A process of acquiring face information of the student while attending the displayed online learning course; a process of performing a face tracking process based on the face information; A process for determining whether or not at least authenticable face information has been acquired by the authentication process; a process of executing authentication multiple times at regular intervals in the online learning, and if the acquisition fails, executing authentication without executing authentication at the timing, and then successfully acquiring face information that can be authenticated by the authentication process, and then executing authentication processing based on the acquired face information and the authenticated face information; A facial recognition method that performs

Images