Method and system for generating and processing virtual teacher expressions and pictures

Abstract

The present invention provides a method and system for generating and processing virtual teacher expressions and pictures. The method comprises: playing electronic courseware, wherein the playing picture of the electronic courseware includes a facial image of a virtual teacher; while playing the electronic courseware, using a front camera of an electronic device where the client is located to collect a facial image of a user who is watching the electronic courseware according to a preset period; binarizing the facial image of the user to obtain a binarized image; and adjusting the facial expression and playing picture of the virtual teacher according to the grayscale value of a pixel point in the binarized image.

Description

Technical Field

[0001] The present invention relates to the field of playback technology, and in particular to a method and system for generating and processing virtual teacher expressions and pictures. Background Art

[0002] Currently, many students use electronic teaching software, which plays electronic courseware through a display screen. The electronic courseware will display some virtual teachers' images to enrich the teaching process and improve the effect of students' acquisition of knowledge.

[0003] Because students stare at the display screen for a long time to watch electronic courseware, it is bound to damage their eyes. If the lighting in the environment where the students are located is uneven, especially the light shining on the students' faces is uneven, it is bound to damage the students' eyes and they cannot use their eyes healthily. Summary of the invention

[0004] The embodiment of the present invention provides a method and system for generating and processing virtual teacher expressions and pictures, so as to provide a technology that can intelligently identify whether the user's eyes are healthy when watching electronic courseware, and correct it in time if it is unhealthy.

[0005] The embodiment of the present invention provides a method for generating and processing virtual teacher expressions and pictures, which is used in a client, and includes:

[0006] Playing the electronic courseware, wherein the playing screen of the electronic courseware includes the facial image of the virtual teacher;

[0007] While playing the electronic courseware, using the front camera of the electronic device where the client is located to collect the facial image of the user who is watching the electronic courseware according to a preset period;

[0008] Binarizing the user's facial image to obtain a binary image;

[0009] The facial expression and the playback picture of the virtual teacher are adjusted according to the grayscale values ​​of the pixels in the binary image.

[0010] In one embodiment, adjusting the facial expression and playback image of the virtual teacher according to the grayscale value of the pixel in the binary image includes the following steps A1-A5:

[0011] Step A1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image;

[0012] Step A2: determine whether the area of ​​the shadow region is equal to or greater than the area of ​​the non-shadow region. If yes, proceed to step A3; otherwise, end the process.

[0013] Step A3, determining a first average grayscale value of the shadow area and a second average grayscale value of the non-shadow area;

[0014] Step A4: Calculate the brightness gradient λ between the shadow area and the non-shadow area according to the following formula (1):

[0015]

[0016] Among them, P min2 is the minimum gray value of the non-shadow area; P max 1 is the maximum gray value of the shadow area; P 2 is the second average gray value; 1 is the first average gray value;

[0017] Step A5, when the light and dark gradient λ is equal to or greater than a preset threshold, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, the light and dark gradient between the latest shadow area and the latest non-shadow area is continuously judged according to a preset period, until it is determined that the light and dark gradient is less than the preset threshold, the electronic courseware is continued to be played, and the facial expression of the virtual teacher is stopped to be displayed with a preset expression; wherein the preset expression includes an angry expression or a sad expression.

[0018] When the brightness gradient λ is less than the preset threshold, the electronic courseware is played normally, and the facial expression of the virtual teacher in the playing picture remains unchanged.

[0019] In one embodiment, adjusting the facial expression and playback image of the virtual teacher according to the grayscale value of the pixel in the binary image includes the following steps B1-B3:

[0020] Step B1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image;

[0021] Step B2, determining whether one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area; if yes, proceed to step B3; otherwise, terminate the process;

[0022] Step B3, when one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, it is started to continuously judge whether the latest shadow area includes one eye image, and whether the latest non-shadow area includes the other eye image according to a preset cycle; until it is determined that the latest shadow area includes one eye image, and the latest non-shadow area includes the other eye image, the playing of the electronic courseware continues, and the display of the facial expression of the virtual teacher with the preset expression is stopped; wherein the preset expression includes an angry expression or a sad expression.

[0023] The embodiment of the present invention also provides a system for generating and processing virtual teacher expressions and pictures, including:

[0024] A playing module, used for playing electronic courseware, wherein the playing screen of the electronic courseware includes a facial image of a virtual teacher;

[0025] A collection module, used for collecting facial images of users watching the electronic courseware according to a preset period by using a front camera of the electronic device where the client is located while playing the electronic courseware;

[0026] A processing module, used for performing binarization processing on the user's facial image to obtain a binarized image;

[0027] The adjustment module is used to adjust the facial expression and playback image of the virtual teacher according to the grayscale value of the pixel points in the binary image.

[0028] In one embodiment, the adjustment module is further configured to perform the following steps A1-A5:

[0029] Step A1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image;

[0030] Step A2: determine whether the area of ​​the shadow region is equal to or greater than the area of ​​the non-shadow region. If yes, proceed to step A3; otherwise, end the process.

[0031] Step A3, determining a first average grayscale value of the shadow area and a second average grayscale value of the non-shadow area;

[0032] Step A4: Calculate the brightness gradient λ between the shadow area and the non-shadow area according to the following formula (1):

[0033]

[0034] Among them, P min2 is the minimum gray value of the non-shadow area; Pmax1 is the maximum gray value of the shadow area; P 2 is the second average gray value; 1 is the first average gray value;

[0035] Step A5, when the light-dark gradient λ determined in step A4 is equal to or greater than a preset threshold, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, the light-dark gradient between the latest shadow area and the latest non-shadow area is continuously determined according to a preset cycle, until it is determined that the light-dark gradient is less than the preset threshold, the electronic courseware is continued to be played, and the facial expression of the virtual teacher is stopped to be displayed with the preset expression; wherein the preset expression includes an angry expression or a sad expression;

[0036] When the brightness gradient λ determined in step A4 is less than the preset threshold, the electronic courseware is played normally, and the facial expression of the virtual teacher in the playing picture remains unchanged.

[0037] In one embodiment, the adjustment module is further used to perform the following steps B1-B3:

[0038] Step B1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image;

[0039] Step B2, determining whether one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area; if yes, proceed to step B3; otherwise, terminate the process;

[0040] Step B3, when one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, it is started to continuously judge whether the latest shadow area includes one eye image, and whether the latest non-shadow area includes the other eye image according to a preset cycle; until it is determined that the latest shadow area includes one eye image, and the latest non-shadow area includes the other eye image, the playing of the electronic courseware continues, and the display of the facial expression of the virtual teacher with the preset expression is stopped; wherein the preset expression includes an angry expression or a sad expression.

[0041] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0042] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0044] Figure 1 The present invention is a flowchart of a method for generating and processing virtual teacher expressions and pictures in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0046] The embodiment of the present invention provides a method for generating and processing virtual teacher expressions and pictures, which is used in a client, such as Figure 1 As shown, steps S1-S4 are included:

[0047] Step S1, play the electronic courseware, and the playing screen of the electronic courseware includes the facial image of the virtual teacher.

[0048] Step S2: While playing the electronic courseware, the front camera of the electronic device where the client is located is used to collect the facial image of the user who is watching the electronic courseware according to a preset period.

[0049] Step S3, binarizing the user's facial image to obtain a binary image;

[0050] Step S4: adjusting the facial expression and playback image of the virtual teacher according to the grayscale values ​​of the pixels in the binary image.

[0051] The above technical solution provided by the embodiment of the present invention adjusts the facial expression of the virtual teacher and the playback screen according to the grayscale value of the pixel in the binary image corresponding to the user's facial image when playing the electronic courseware; wherein, the grayscale value of the pixel in the binary image can reflect the lighting information actually received by the user's face, therefore, using this information to adjust the playback screen can make the playback screen match the lighting information actually received by the user's face, so that the user feels more comfortable when viewing the playback screen, and reduces the user's eye fatigue. In addition, adjusting the facial expression of the virtual teacher can remind the user that the lighting conditions of the current environment are not conducive to the user's eyes. Under this reminder, the user can adjust the lighting conditions of the current environment according to his own wishes, so that the user can watch the electronic courseware in a more comfortable environment, thereby realizing intelligent recognition of whether the user's eyes are healthy when watching the electronic courseware, and timely correction when it is unhealthy.

[0052] In one embodiment, step S4 "adjusting the facial expression and playback image of the virtual teacher according to the grayscale value of the pixel points in the binary image" can be implemented as the following steps A1-A4:

[0053] Step A1: determining the shadow area and non-shadow area of ​​the face part in the facial image according to the grayscale values ​​of the pixels in the binary image.

[0054] Step A2: determine whether the area of ​​the shadow region is equal to or greater than the area of ​​the non-shadow region. If yes, proceed to step A3; otherwise, end the process.

[0055] Step A3, determining a first average gray value of the shadow area and a second average gray value of the non-shadow area;

[0056] Step A4: Calculate the brightness gradient λ between the shadow area and the non-shadow area according to the following formula (1):

[0057]

[0058] Among them, P min2 is the minimum gray value of the non-shadow area; P max1 is the maximum gray value of the shadow area; P 2 is the second average gray value; 1 is the first average gray value;

[0059] Step A5. When the light-dark gradient λ determined in step A4 is equal to or greater than a preset threshold, it indicates that the user is in a light environment where the illumination on the user's face is particularly uneven. At this time, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, the light-dark gradient between the latest shadow area and the latest non-shadow area is continuously judged according to a preset cycle until it is determined that the light-dark gradient is less than the preset threshold, then the playing of the electronic courseware continues, and the display of the facial expression of the virtual teacher with the preset expression is stopped; wherein the preset expression includes an angry expression or a sad expression.

[0060] When the brightness gradient λ determined in step A4 is less than the preset threshold, it indicates that the user is in a light environment where the illumination on the user's face is relatively uniform. At this time, the electronic courseware is played normally, and the facial expression of the virtual teacher in the playback screen remains unchanged.

[0061] The above technical solution can intelligently identify whether the user is in an environment with uneven lighting. When it is identified that the user is indeed in an environment with uneven lighting, the facial expression of the virtual teacher is adjusted to a preset expression and the playing of the electronic courseware is paused to achieve the purpose of reminding the user to adjust the lighting environment. When the user adjusts the lighting environment and eliminates the uneven lighting, the electronic courseware is played normally and the normal expression of the virtual teacher is displayed intelligently and automatically. This technical solution realizes the intelligent reminder of the user to adjust the lighting conditions and ensure the healthy use of the user's eyes.

[0062] In one embodiment, step S4 "adjusting the facial expression and playback image of the virtual teacher according to the grayscale value of the pixel points in the binary image" can be implemented as the following steps B1-B3:

[0063] Step B1: determining the shadow area and non-shadow area of ​​the face part in the facial image according to the grayscale values ​​of the pixels in the binary image.

[0064] Step B2, determine whether one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area; if yes, proceed to step B3; otherwise, end the process.

[0065] Step B3, when one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, it is started to continuously judge whether the latest shadow area includes one eye image, and whether the latest non-shadow area includes the other eye image according to a preset cycle; until it is determined that the latest shadow area includes one eye image, and the latest non-shadow area includes the other eye image, the playing of the electronic courseware continues, and the display of the facial expression of the virtual teacher with the preset expression is stopped; wherein the preset expression includes an angry expression or a sad expression.

[0066] The above technical solution can intelligently identify whether the user is in an environment with uneven lighting. When it is identified that the user is indeed in an environment with uneven lighting, the facial expression of the virtual teacher is adjusted to a preset expression and the playing of the electronic courseware is paused to achieve the purpose of reminding the user to adjust the lighting environment. When the user adjusts the lighting environment and eliminates the uneven lighting, the electronic courseware is played normally and the normal expression of the virtual teacher is displayed intelligently and automatically. This technical solution realizes the intelligent reminder of the user to adjust the lighting conditions and ensure the healthy use of the user's eyes.

[0067] Corresponding to the above method provided by the embodiment of the present invention, the embodiment of the present invention further provides a system for generating and processing virtual teacher expressions and pictures, including:

[0068] A playing module, used for playing electronic courseware, wherein the playing screen of the electronic courseware includes a facial image of a virtual teacher;

[0069] A collection module, used for collecting facial images of users watching the electronic courseware according to a preset period by using a front camera of the electronic device where the client is located while playing the electronic courseware;

[0070] A processing module, used for performing binarization processing on the user's facial image to obtain a binarized image;

[0071] The adjustment module is used to adjust the facial expression and playback image of the virtual teacher according to the grayscale value of the pixel points in the binary image.

[0072] In one embodiment, the adjustment module is further configured to perform the following steps A1-A5:

[0073] Step A1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image;

[0074] Step A2: determine whether the area of ​​the shadow region is equal to or greater than the area of ​​the non-shadow region. If yes, proceed to step A3; otherwise, end the process.

[0075] Step A3, determining a first average grayscale value of the shadow area and a second average grayscale value of the non-shadow area;

[0076] Step A4: Calculate the brightness gradient λ between the shadow area and the non-shadow area according to the following formula (1):

[0077]

[0078] Among them, P min2 is the minimum gray value of the non-shadow area; P max1 is the maximum gray value of the shadow area; P 2 is the second average gray value; 1 is the first average gray value;

[0079] Step A5, when the light-dark gradient λ determined in step A4 is equal to or greater than a preset threshold, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, the light-dark gradient between the latest shadow area and the latest non-shadow area is continuously determined according to a preset cycle, until it is determined that the light-dark gradient is less than the preset threshold, the electronic courseware is continued to be played, and the facial expression of the virtual teacher is stopped to be displayed with the preset expression; wherein the preset expression includes an angry expression or a sad expression;

[0080] When the brightness gradient λ determined in step A4 is less than the preset threshold, the electronic courseware is played normally, and the facial expression of the virtual teacher in the playing picture remains unchanged.

[0081] In one embodiment, the adjustment module is further used to perform the following steps B1-B3:

[0082] Step B1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image;

[0083] Step B2, determining whether one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area; if yes, proceed to step B3; otherwise, terminate the process;

[0084] Step B3, when one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, it is started to continuously judge whether the latest shadow area includes one eye image, and whether the latest non-shadow area includes the other eye image according to a preset cycle; until it is determined that the latest shadow area includes one eye image, and the latest non-shadow area includes the other eye image, the playing of the electronic courseware continues, and the display of the facial expression of the virtual teacher with the preset expression is stopped; wherein the preset expression includes an angry expression or a sad expression.

[0085] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for generating and processing virtual teacher expressions and pictures, used in the client, It is characterized in that include: Playing the electronic courseware, wherein the playing screen of the electronic courseware includes the facial image of the virtual teacher; While playing the electronic courseware, using the front camera of the electronic device where the client is located to collect the facial image of the user who is watching the electronic courseware according to a preset period; Binarizing the user's facial image to obtain a binary image; According to the grayscale values ​​of the pixels in the binary image, the facial expression and the playback image of the virtual teacher are adjusted; The step of adjusting the facial expression and the playback image of the virtual teacher according to the grayscale value of the pixel in the binary image includes the following steps B1-B3: Step B1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image; Step B2, determining whether one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area; if yes, proceed to step B3; otherwise, terminate the process; Step B3, when one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, it is started to continuously judge whether the latest shadow area includes one eye image, and whether the latest non-shadow area includes the other eye image according to a preset cycle; until it is determined that the latest shadow area includes one eye image, and the latest non-shadow area includes the other eye image, the playing of the electronic courseware continues, and the display of the facial expression of the virtual teacher with the preset expression is stopped; wherein the preset expression includes an angry expression or a sad expression.

2. The method according to claim 1, It is characterized in that The step of adjusting the facial expression and the playback image of the virtual teacher according to the grayscale value of the pixel in the binary image comprises the following steps A1-A5: Step A1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image; Step A2: determine whether the area of ​​the shadow region is equal to or greater than the area of ​​the non-shadow region. If yes, proceed to step A3; otherwise, end the process. Step A3, determining a first average grayscale value of the shadow area and a second average grayscale value of the non-shadow area; Step A4: Calculate the brightness gradient λ between the shadow area and the non-shadow area according to the following formula (1): λ=P min2 -P max1 (1) P 2 -P 1 Among them, P min2 is the minimum gray value of the non-shadow area; P max1 is the maximum gray value of the shadow area; P 2 is the second average gray value; 1 is the first average gray value; Step A5, when the light-dark gradient λ is equal to or greater than a preset threshold, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, the light-dark gradient between the latest shadow area and the latest non-shadow area is continuously determined according to a preset cycle, until it is determined that the light-dark gradient is less than the preset threshold, the electronic courseware is continued to be played, and the facial expression of the virtual teacher is stopped to be displayed with the preset expression; wherein the preset expression includes an angry expression or a sad expression; When the brightness gradient λ is less than the preset threshold, the electronic courseware is played normally, and the facial expression of the virtual teacher in the playing picture remains unchanged.

3. A system for generating and processing virtual teacher expressions and images, It is characterized in that include: A playing module, used for playing electronic courseware, wherein the playing screen of the electronic courseware includes a facial image of a virtual teacher; A collection module, used for collecting facial images of users watching the electronic courseware according to a preset period by using a front camera of the electronic device where the client is located while playing the electronic courseware; A processing module, used for performing binarization processing on the user's facial image to obtain a binarized image; An adjustment module, used for adjusting the facial expression and playback picture of the virtual teacher according to the grayscale value of the pixel points in the binary image; The adjustment module is further used to perform the following steps B1-B3: Step B1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image; Step B2, determining whether one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area; if yes, proceed to step B3; otherwise, terminate the process; Step B3, when one eye image in the binary image is located in the shadow area and the other eye image is located in the non-shadow area, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, it is started to continuously judge whether the latest shadow area includes one eye image, and whether the latest non-shadow area includes the other eye image according to a preset cycle; until it is determined that the latest shadow area includes one eye image, and the latest non-shadow area includes the other eye image, the playing of the electronic courseware continues, and the display of the facial expression of the virtual teacher with the preset expression is stopped; wherein the preset expression includes an angry expression or a sad expression.

4. The system according to claim 3, It is characterized in that The adjustment module is further used to perform the following steps A1-A5: Step A1, determining the shadow area and non-shadow area of ​​the face according to the grayscale values ​​of the pixels in the binary image; Step A2: determine whether the area of ​​the shadow region is equal to or greater than the area of ​​the non-shadow region. If yes, proceed to step A3; otherwise, end the process. Step A3, determining a first average grayscale value of the shadow area and a second average grayscale value of the non-shadow area; Step A4: Calculate the brightness gradient λ between the shadow area and the non-shadow area according to the following formula (1): Among them, P min2 is the minimum gray value of the non-shadow area; P max1 is the maximum gray value of the shadow area; P 2 is the second average gray value; 1 is the first average gray value; Step A5, when the light-dark gradient λ determined in step A4 is equal to or greater than a preset threshold, the facial expression of the virtual teacher is adjusted to a preset expression, and the playing of the electronic courseware is paused; at the same time, the light-dark gradient between the latest shadow area and the latest non-shadow area is continuously determined according to a preset cycle, until it is determined that the light-dark gradient is less than the preset threshold, the electronic courseware is continued to be played, and the facial expression of the virtual teacher is stopped to be displayed with the preset expression; wherein the preset expression includes an angry expression or a sad expression; When the brightness gradient λ determined in step A4 is less than the preset threshold, the electronic courseware is played normally, and the facial expression of the virtual teacher in the playing picture remains unchanged.

Images