Classroom teaching auxiliary method, system and device

Abstract

The invention relates to a classroom teaching auxiliary method, system and device. The method comprises the following steps: acquiring a speckle image generated after a laser scans a classroom; determining an imaging image according to the speckle image; determining the state information of the student according to the imaging image; and performing auxiliary teaching on the classroom according to each piece of state information. Laser imaging is used and is not affected by classroom ambient light, laser imaging is thermal imaging, complete image information of people cannot be obtained, and even if hackers invade, personal privacy of students cannot be obtained. In addition, the state information of the students is obtained through the imaged images, auxiliary teaching can be carried out on the classroom through the state information, and the purpose of auxiliary monitoring of classroom teaching is achieved.

Description

Background Art

[0002] The monitoring of classroom teaching effects and students' learning status has always been an important auxiliary means in classroom teaching. Traditional monitoring is mainly achieved through cameras, which mainly analyze students' states in the classroom by identifying facial features and combining learning-related behaviors such as reading, raising hands, writing, etc. in class, and then combining facial expressions. However, there are also some problems with this analysis. For example, cameras are greatly affected by ambient light. Especially when using projection teaching, the ambient light in the classroom is weak, and the projection brightness changes with the playback time of the courseware, which will affect the imaging quality. Cameras obtain complete image information of people, with a large amount of data, and it is easy for hackers to obtain students' personal privacy. Summary of the Invention

[0003] In order to overcome the problems that the image of students captured by a visible light camera is greatly affected by the classroom ambient light and it is easy to leak students' personal privacy, the present invention provides a classroom teaching assistance method, system and device.

[0004] In a first aspect, to solve the above technical problems, the present invention provides a classroom teaching assistance method, including:

[0005] Obtaining a speckle image generated after a laser scans the classroom;

[0006] Determining an imaging image according to the speckle image;

[0007] Determining students' status information according to the imaging image;

[0008] Carrying out auxiliary teaching for the classroom according to each status information.

[0009] In a second aspect, the present invention provides a classroom teaching assistance system, including:

[0010] A speckle image acquisition module for obtaining a speckle image generated after a laser scans the classroom;

[0011] An imaging image determination module for determining an imaging image according to the speckle image;

[0012] A status information determination module for determining students' status information according to the imaging image;

[0013] An auxiliary teaching module for carrying out auxiliary teaching for the classroom according to each status information.

[0014] In a third aspect, the present invention provides a classroom teaching assistance device, comprising: a mid-infrared laser, a near-infrared laser, a controller module, a digital micromirror, a photodetector, a correlation calculation module, an edge analysis module, a thermal analysis imaging module, and a carbon dioxide imaging analysis module; the controller module is connected to the digital micromirror, the photodetector is connected to the correlation calculation module, and the correlation calculation module is respectively connected to the edge analysis module, the thermal analysis imaging module, and the carbon dioxide imaging analysis module;

[0015] The correlation calculation module is used to execute a classroom teaching assistance method.

[0016] The beneficial effects of the present invention are as follows: The present application uses laser imaging, which is not affected by the classroom environment light, and the laser imaging is thermal imaging, and does not obtain the complete image information of people. Even if it is hacked, the personal privacy of students cannot be obtained. In addition, by obtaining the status information of students from the imaging images, the classroom can be assisted in teaching through the status information, so as to achieve the purpose of assisting in monitoring classroom teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be further described below with reference to the drawings and embodiments.

[0018] Figure 1 It is a schematic flowchart of a classroom teaching assistance method according to an embodiment of the present invention;

[0019] Figure 2 It is a schematic structural diagram of a classroom teaching assistance system according to an embodiment of the present invention;

[0020] Figure 3 It is a schematic structural diagram of a classroom teaching assistance device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The following embodiments are further explanations and supplements to the present invention and do not constitute any limitation to the present invention.

[0022] The following describes a classroom teaching assistance method, system, and device according to an embodiment of the present invention with reference to the drawings.

[0023] As Figure 1 shown, a classroom teaching assistance method according to an embodiment of the present invention includes:

[0024] S1. Obtain the speckle image generated after the laser scans the classroom.

[0025] S2. Determine the imaging image according to the speckle image.

[0026] S3. Determine the status information of the students according to the imaging image.

[0027] S4. Assist teaching in the classroom according to each status information.

[0028] In this embodiment, a laser is used for imaging, which is not affected by the light in the classroom environment. Moreover, the laser imaging is thermal imaging and does not obtain the complete image information of people. Even if hacked, the personal privacy of students cannot be obtained. In addition, by obtaining the status information of students from the imaging images, the classroom can be assisted in teaching according to the status information, achieving the purpose of auxiliary monitoring of classroom teaching.

[0029] In this embodiment, the status information of students can be obtained according to the actual situation. For example, in a 40-minute class, the information status of students in the classroom can be monitored at 10 minutes, 20 minutes, and 30 minutes respectively, or the information status of students in the classroom can be monitored every 5 minutes, or when spot-checking, the information status of students in the classroom can be monitored in real time.

[0030] Optionally, obtaining the speckle images generated after the laser scans the classroom includes:

[0031] Controlling the digital micromirror to be at a preset angle through the controller module;

[0032] After expanding the beam of the mid-infrared laser, irradiate the laser on the digital micromirror to scan the classroom to determine the first speckle image;

[0033] After expanding the beam of the near-infrared laser, irradiate the laser on the digital micromirror to scan the classroom to determine the second speckle image;

[0034] The speckle images include the first speckle image and the second speckle image.

[0035] In this embodiment, the mid-infrared laser can be a carbon dioxide laser. The carbon dioxide laser imaging can obtain information related to students' breathing. Imaging light of these wavelengths is easier to obtain comprehensive information about the classroom.

[0036] In this embodiment, the near-infrared laser can be a gallium arsenide near-infrared laser. The gallium arsenide near-infrared laser has high efficiency and low cost. The infrared light is invisible and will not cause adverse effects on teaching. Moreover, the infrared light has strong penetration ability and is not easily affected by dust, etc.

[0037] In this embodiment, the laser is equipped with a beam expander. After beam expansion, the laser spot size is large, which can cover the digital micromirror and expand the monitoring range.

[0038] Lights of different wavelengths have their own imaging advantages. For example, near-infrared light has strong anti-background light interference ability, mid-infrared imaging can more effectively obtain information such as temperature, and has strong penetration ability and is less affected by dust.

[0039] In this embodiment, the digital micromirror is controlled by a controller module and can rotate freely. Moreover, the digital micromirror can modulate visible light, near-infrared light, and mid-infrared light simultaneously, with high modulation speed and high resolution.

[0040] Optionally, determining the imaging image according to the speckle image includes:

[0041] Converting the first speckle image into a first electrical signal through a photodetector;

[0042] Converting the second speckle image into a second electrical signal through a photodetector

[0043] Determining the first imaging image according to the correlation between the first speckle image and the first electrical signal;

[0044] Determining the second imaging image according to the correlation between the second speckle image and the second electrical signal;

[0045] The imaging image includes the first imaging image and the second imaging image.

[0046] In this embodiment, the photodetectors are indium antimonide photodetectors and mercury cadmium telluride photodetectors. Among them, the indium antimonide photodetectors and mercury cadmium telluride photodetectors can efficiently convert the second speckle image and the first speckle image into electrical signals respectively.

[0047] In this embodiment, correlation imaging is performed by combining two or more light sources with different wavelengths, a digital micromirror, and a photodetector, which can facilitate multi-wavelength imaging.

[0048] Optionally, determining the imaging image according to the speckle image includes:

[0049] Determining the image matrix according to the correlation between the speckle image and the electrical signal. The formula is as follows:

[0050] ΔG(x,y) = <(I1 - <i1>)[I2(x,y) - <I2(x,y)>]>;

[0051] Wherein, if I1 represents the first electrical signal and I2(x,y) represents the first speckle image, then ΔG(x,y) represents the first image matrix corresponding to the mid-infrared laser; if I1 represents the second electrical signal and I2(x,y) represents the second speckle image, then ΔG(x,y) represents the second image matrix corresponding to the near-infrared laser;

[0052] The imaging image is determined by reconstructing the image matrix through a compressive sensing algorithm.

[0053] Since the speckle image cannot be directly used to identify the state information of students, it is necessary to reconstruct the speckle image to obtain a clearer classroom image, so as to reflect the state information of students in the classroom. Therefore, the reconstruction of the classroom image can be realized according to the correlation between the speckle image generated by the digital micromirror and the electrical signal converted by the photodetector.

[0054] Optionally, determining the state information of students according to the imaging image includes:

[0055] Uploading the first imaging image to the thermal analysis imaging module and the carbon dioxide imaging analysis module respectively, and uploading the second imaging image to the edge analysis module;

[0056] And determining the temperature information of students by the thermal analysis imaging module according to the first imaging image;

[0057] And determining the carbon dioxide change information of students by the carbon dioxide imaging analysis module according to the first imaging image;

[0058] And determining the edge information of students by the edge analysis module according to the second imaging image;

[0059] The state information includes temperature information, carbon dioxide change information and edge information.

[0060] In this embodiment, the temperature information of students is determined by the thermal analysis imaging module, and the concentration of students' attention can be judged. For example, when concentrating, adrenaline will rise, which will cause the body temperature regulation center to increase the body temperature.

[0061] In this embodiment, the carbon dioxide change information of students is determined by the carbon dioxide imaging analysis module. When students are thinking hard, they will consume more energy and exhale more carbon dioxide.

[0062] In this embodiment, the edge information of students is determined by the edge analysis module, which can effectively analyze classroom behaviors such as reading, raising hands, and writing, and can also obtain facial expression information, but will not obtain personal images of people and will not disclose personal privacy.

[0063] Optionally, the thermal analysis imaging module determines the temperature information of the student according to the first imaging image, including:

[0064] Obtain the change of the first image matrix within a preset time period;

[0065] Compare the change with a preset value to determine the temperature information.

[0066] Since the first imaging image is an infrared image, and the first image matrix can well reflect the temperature change of the student, and the human body temperature is generally constant and only rises when the attention is concentrated. Based on the above, obtaining the change of the first image matrix of the student within a preset time period and comparing the change with a preset value can determine the temperature information.

[0067] For example, the preset value is 36.5 °C, and the temperature of the first image matrix per minute within 5 minutes is 37 °C, 37.1 °C, 37 °C, 37.2 °C and 37 °C, which indicates that the student's adrenaline has risen and the attention is concentrated.

[0068] In this embodiment, the temperature information obtained is the overall temperature information of the first imaging image. During monitoring, the temperature information corresponding to the student can be found on the first imaging image according to the position of each student, so as to judge the status information of the student.

[0069] Optionally, the carbon dioxide imaging analysis module determines the carbon dioxide change information of the student according to the first imaging image, including:

[0070] Calculate the contrast-to-noise ratio of the second imaging image according to the second image matrix;

[0071] Take the contrast-to-noise ratio as the carbon dioxide change information.

[0072] In this embodiment, since carbon dioxide has a strong absorption effect on the laser emitted by the carbon dioxide laser, the carbon dioxide exhaled by people will cause an increase in the noise of the image near the human face. Thus, the change of carbon dioxide exhaled by people can be obtained by analyzing the noise of the first imaging image.

[0073] The calculation process of the contrast-to-noise ratio is as follows:

[0074] For the student, calculate the first correlation value corresponding to the first image matrix at the first position corresponding to the first preset reflectivity, and the second correlation value corresponding to the first image matrix at the second position corresponding to the second preset reflectivity;

[0075] For the student, determine the contrast-to-noise ratio according to the first correlation value, the second correlation value, the first preset variance at the first position and the second preset variance at the second position.

[0076] The formula is as follows:

[0077]

[0078] Among them, CNR represents the contrast-to-noise ratio, and ΔG1 and ΔG0 respectively represent the first correlation value and the second correlation value. respectively represent the first preset variance and the second preset variance.

[0079] In this embodiment, the first preset reflectivity is 1 and the second preset reflectivity is 0.

[0080] In this embodiment, the obtained carbon dioxide change situation is the carbon dioxide change situation of the entire first imaging image. During monitoring, the carbon dioxide change situation corresponding to each student can be found on the first imaging image according to the position of each student, so as to judge the status information of the student.

[0081] Optionally, the edge analysis module determines the edge information of the student according to the second imaging image, including:

[0082] Reconstruct the edge of the object in the second imaging image through the speckle drift ghost imaging algorithm to determine the edge information.

[0083] In this embodiment, the speckle drift ghost imaging algorithm is the SSGI algorithm (Screen-Space Global Illumination, a graphics algorithm implemented based on screen space). This SSGI algorithm is a prior art and will not be elaborated here.

[0084] As Figure 2 shown, the present invention provides a classroom teaching assistance system, including:

[0085] A speckle image acquisition module, configured to acquire a speckle image generated after a laser scans a classroom;

[0086] An imaging image determination module, configured to determine an imaging image according to the speckle image;

[0087] A status information determination module, configured to determine the status information of a student according to the imaging image;

[0088] An auxiliary teaching module, configured to assist in teaching the classroom according to each status information.

[0089] Optionally, the speckle image acquisition module is specifically configured to:

[0090] Control the digital micromirror to be at a preset angle through the controller module;

[0091] Expand the mid-infrared laser and irradiate the laser on the digital micromirror to scan the classroom to determine the first speckle image;

[0092] After expanding the beam of a near-infrared laser, the laser is irradiated on a digital micromirror to scan the classroom to determine a second speckle image;

[0093] The speckle image includes a first speckle image and a second speckle image.

[0094] Optionally, the imaging image determination module is specifically configured to:

[0095] Convert the first speckle image into a first electrical signal through a photodetector;

[0096] Convert the second speckle image into a second electrical signal through a photodetector

[0097] Determine a first imaging image according to the correlation between the first speckle image and the first electrical signal;

[0098] Determine a second imaging image according to the correlation between the second speckle image and the second electrical signal;

[0099] The imaging image includes a first imaging image and a second imaging image.

[0100] Optionally, the imaging image determination module is specifically configured to:

[0101] Determine the imaging image according to the speckle image, including:

[0102] Determine an image matrix according to the correlation between the speckle image and the electrical signal. The formula is as follows:

[0103] ΔG(x,y)=<(I1- <i1>)[I2(x,y)-<I(x,y)>]>;

[0104] Wherein, if I1 represents the first electrical signal and I2(x, y) represents the first speckle image, then ΔG(x, y) represents the first image matrix corresponding to the mid-infrared laser; if I1 represents the second electrical signal and I2(x, y) represents the second speckle image, then ΔG(x, y) represents the second image matrix corresponding to the near-infrared laser;

[0105] The imaging image is determined by reconstructing the image matrix through the compressive sensing algorithm.

[0106] Optionally, the status information determination module is specifically configured to:

[0107] Upload the first imaging image to the thermal analysis imaging module and the carbon dioxide imaging analysis module respectively, and upload the second imaging image to the edge analysis module;

[0108] And determine the temperature information of the student through the thermal analysis imaging module according to the first imaging image;

[0109] And determine the carbon dioxide change information of the student through the carbon dioxide imaging analysis module according to the first imaging image;

[0110] And determine the edge information of the student through the edge analysis module according to the second imaging image;

[0111] The status information includes temperature information, carbon dioxide change information and edge information.

[0112] Optionally, the status information determination module is specifically configured to:

[0113] Obtain the change situation of the first image matrix within a preset time period;

[0114] Compare the change situation with a preset value to determine the temperature information.

[0115] Optionally, the status information determination module is specifically configured to:

[0116] Calculate the contrast-to-noise ratio of the first imaging image according to the first image matrix;

[0117] Use the contrast-to-noise ratio as the carbon dioxide change information.

[0118] Optionally, the status information determination module is specifically configured to:

[0119] Determine the edge information by reconstructing the edge of the object in the second imaging image through the speckle drift ghost imaging algorithm.

[0120] Optionally, as Figure 3 As shown, a classroom teaching assistance device includes: a mid-infrared laser, a near-infrared laser, a controller module, a digital micromirror, a photodetector, an association calculation module, an edge analysis module, a thermal analysis imaging module, and a carbon dioxide imaging analysis module; the controller module is connected to the digital micromirror, the photodetector is connected to the association calculation module, and the association calculation module is respectively connected to the edge analysis module, the thermal analysis imaging module, and the carbon dioxide imaging analysis module;

[0121] The association calculation module is used to execute a classroom teaching assistance method.

[0122] Those skilled in the art of the present technology know that the present invention can be implemented as a system, a method, or a computer program product. Therefore, the present disclosure can be specifically implemented in the following forms, that is: it can be completely hardware, can also be completely software (including firmware, resident software, microcode, etc.), or can also be in the form of a combination of hardware and software, which is generally referred to as "circuit", "module", or "system" in this article. In addition, in some embodiments, the present invention can also be implemented in the form of a computer program product in one or more computer-readable media, which contains computer-readable program code. The computer-readable storage medium can be, for example, but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above.

[0123] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0124] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be construed as limitations on the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A classroom teaching assistance method, characterized in that, Including: Obtaining a speckle image generated after a laser scans a classroom; Determining an imaging image according to the speckle image; Determining the status information of students according to the imaging image; Assisting in teaching the class according to each of the status information.

2. The method according to claim 1, wherein The obtaining of the speckle image generated after the laser scans the classroom includes: Controlling a digital micromirror to be at a preset angle through a controller module; Expanding a mid-infrared laser and irradiating the laser on the digital micromirror to scan the classroom to determine a first speckle image; Expanding a near-infrared laser and irradiating the laser on the digital micromirror to scan the classroom to determine a second speckle image; The speckle image includes the first speckle image and the second speckle image.

3. The method according to claim 2, characterized in that, Determining the imaging image according to the speckle image includes: Converting the first speckle image into a first electrical signal through a photodetector; Converting the second speckle image into a second electrical signal through a photodetector Determining a first imaging image according to the correlation relationship between the first speckle image and the first electrical signal; Determining a second imaging image according to the correlation relationship between the second speckle image and the second electrical signal; The imaging image includes the first imaging image and the second imaging image.

4. The method according to claim 3, wherein Determining the imaging image according to the speckle image includes: Determining an image matrix according to the correlation relationship between the speckle image and the electrical signal, and the formula is as follows: ΔG(x, y) = <(I1 - <i1>)[I2(x, y)-<I2(x, y)>]>; Wherein, if I1 represents the first electrical signal and I2(x, y) represents the first speckle image, then ΔG(x, y) represents the first image matrix corresponding to the mid-infrared laser, and if I1 represents the second electrical signal and I2(x, y) represents the second speckle image, then ΔG(x, y) represents the second image matrix corresponding to the near-infrared laser; Reconstructing the image matrix through a compressive sensing algorithm to determine the imaging image.

5. The method according to claim 4, characterized in that The determining of the status information of students according to the imaging image includes: Uploading the first imaging image to a thermal analysis imaging module and a carbon dioxide imaging analysis module respectively, and uploading the second imaging image to an edge analysis module; And determining the temperature information of students through the thermal analysis imaging module according to the first imaging image; And determining the carbon dioxide change information of students through the carbon dioxide imaging analysis module according to the first imaging image; And determining the edge information of students through the edge analysis module according to the second imaging image; The status information includes the temperature information, the carbon dioxide change information and the edge information.

6. The method according to claim 5, characterized in that, The determining of the temperature information of students through the thermal analysis imaging module according to the first imaging image includes: Obtaining the change situation of the first image matrix within a preset time period; Comparing the change situation with a preset value to determine the temperature information.

7. The method according to claim 5, characterized in that The determining of the carbon dioxide change information of students through the carbon dioxide imaging analysis module according to the first imaging image includes: Calculating the contrast-to-noise ratio of the first imaging image according to the first image matrix; Taking the contrast-to-noise ratio as the carbon dioxide change information.

8. The method according to claim 5, characterized in that The determining of the edge information of students through the edge analysis module according to the second imaging image includes: Determine the edge information by reconstructing the edge of the object in the second imaging image through the speckle drift ghost imaging algorithm.

9. A classroom teaching assistance system, characterized in that, It includes: A speckle image acquisition module for acquiring the speckle image generated after the laser scans the classroom. An imaging image determination module for determining the imaging image according to the speckle image. A status information determination module for determining the status information of the students according to the imaging image. An auxiliary teaching module for assisting classroom teaching according to each of the status information.

10. A classroom teaching aid device, applied to a classroom teaching aid method as described in claim 5, characterized in that, It includes: A mid-infrared laser, a near-infrared laser, a controller module, a digital micromirror, a photodetector, a correlation calculation module, an edge analysis module, a thermal analysis imaging module, and a carbon dioxide imaging analysis module; the controller module is connected to the digital micromirror, the photodetector is connected to the correlation calculation module, and the correlation calculation module is respectively connected to the edge analysis module, the thermal analysis imaging module, and the carbon dioxide imaging analysis module. The correlation calculation module is used to execute a classroom teaching assistance method.

Images