A super-resolution reconstruction method based on blackboard features in online classroom scenarios

Abstract

The present invention discloses a super-resolution reconstruction method based on blackboard features in an online classroom scenario. The steps of the super-resolution reconstruction method are as follows: a blackboard photo is taken by a camera set up in the classroom, and the blackboard writing area is accurately extracted using a target detection model, and then image super-resolution reconstruction processing is performed. In addition, for the blackboard writing content with obvious edge features, gradient constraints are introduced in the super-resolution model to strengthen feature learning, effectively improve the resolution of the blackboard writing content, thereby improving the students' learning efficiency in class. The method has the characteristics of fast running speed, low cost, and easy use, ensuring the smooth progress of online live teaching in campus classrooms.

Description

Technical Field

[0001] The present invention relates to the technical field of image super-resolution reconstruction, and in particular to a super-resolution reconstruction method based on blackboard features in an online classroom scenario. Background Art

[0002] With the popularization of Internet technology, online education is quietly changing the way people learn, and more and more people are participating in online live teaching in campus classrooms through the Internet. At the same time, with the increase in the number of participants and the improvement in the requirements for video quality, the real-time performance required for live teaching is challenged under limited network bandwidth. In addition, due to factors such as shooting angle or distance, non-knowledge content areas such as podiums and walls will appear, which is not friendly when bandwidth is limited. With the development of image processing technology, the traditional image calibration method can be used to crop the blackboard area, and the downsampling method can be used to compress the transmitted information and use image super-resolution technology to reconstruct the image information at the sender to reduce the amount of information that needs to be transmitted, thereby reducing the demand for network bandwidth.

[0003] However, some classrooms are equipped with movable blackboards, and in online open classes, the camera generally moves with the teacher's body. The traditional image calibration method cannot accurately extract the blackboard area, which will not only make the final image reconstruction index affected by irrelevant factors, but also cause a waste of equipment computing power. In addition, the chalk characters on the blackboard have obvious edge features, but most of the existing super-resolution models are not designed for edge features, and the existing super-resolution reconstruction technology pays less attention to the reconstruction speed, which makes it difficult to embed and use in scenarios with high real-time requirements such as remote live teaching. Summary of the invention

[0004] The purpose of the present invention is to solve the above-mentioned defects in the prior art and provide a super-resolution reconstruction method based on blackboard features in an online classroom scenario. The method accurately extracts the blackboard writing area by improving the target detection model YOLOv5, reduces the waste of computing power caused by the reconstruction of non-blackboard areas, and then uses a super-resolution model with gradient constraints to reconstruct the extracted blackboard writing area, which can timely and effectively improve the resolution of the blackboard writing content in the photo.

[0005] The purpose of the present invention can be achieved by adopting the following technical solutions:

[0006] A super-resolution reconstruction method based on blackboard features in an online classroom scenario, the super-resolution reconstruction method comprising the following steps:

[0007] S1, build blackboard high-resolution and low-resolution datasets for online classroom scenarios;

[0008] S2. Build a target detection model YOLOv5, and detect the input image through the target detection model YOLOv5, and use non-maximum suppression to output the four coordinate values ​​of the minimum circumscribed rectangle containing the blackboard area, so as to crop the blackboard area in the input image; reduce the reconstruction area of ​​the image super-resolution model, thereby reducing the calculation amount of the image super-resolution model and avoiding the interference of the non-blackboard area on the reconstruction effect of the super-resolution model;

[0009] S3, constructing an image super-resolution model that introduces gradient constraints, and reconstructing the blackboard writing area with obvious edge features through the image super-resolution model to obtain a corresponding super-resolution reconstructed image;

[0010] S4. Perform bicubic upsampling interpolation on the low-resolution image input in step S2, and then integrate it with the reconstructed image of the blackboard area obtained in step S3, so as to output a super-resolution reconstruction result based on blackboard features in the online classroom scenario.

[0011] Furthermore, the process of step S1 is as follows:

[0012] Use a camera to record online classroom videos, or collect online open class recordings through web crawlers. Then, extract frames from the video at intervals to obtain a series of blackboard images. Use the series of blackboard images as the blackboard high-resolution dataset for the online classroom scene, and use the bicubic downsampling interpolation method to obtain the corresponding blackboard low-resolution dataset, which is then divided into training dataset and test dataset in a ratio of 7:3. Using the dataset corresponding to the target scene for model training is conducive to improving the model's reconstruction effect on the blackboard area.

[0013] Furthermore, the target detection model YOLOv5 includes a feature extraction module, a feature fusion module and a feature prediction module which are connected in sequence.

[0014] First, the feature extraction module is composed of a Focus module and three CSP modules connected in sequence. The Focus module contains four parallel slicing layers and a convolution layer. After slicing the input image, it is spliced ​​in the channel dimension and convolved. The outputs of the three CSP modules are recorded as feature map M1, feature map M2 and feature map M3 in order. The feature extraction module divides the data into 4 parts, each of which is equivalent to 2 times downsampling, and then splices them in the channel dimension, which can improve the receptive field of each pixel and minimize information loss.

[0015] Then, the feature fusion module is composed of a feature pyramid module FPN and a path aggregation module PAN. The feature pyramid module FPN uses an upsampling method to fuse information. The feature map M3 is first upsampled, and then fused with the feature map M2 to obtain the feature map F1. The feature map F1 is then upsampled and fused with the feature map M1 to obtain the feature map F2. The path aggregation module PAN uses a bottom-up method to fuse features. The feature map F2 first passes through a convolution layer with a step size of 2, and then is fused with the feature map F1 to obtain the feature map P1. The feature map P1 passes through a convolution layer with a step size of 2, and then is fused with the feature map M3 to obtain the feature map P2. The feature fusion module uses the feature pyramid module FPN to convey strong semantic features from top to bottom, and uses the path aggregation module PAN to convey strong positioning features from bottom to top, and performs feature fusion on different feature maps from different dimensions.

[0016] Finally, the feature prediction module uses non-maximum suppression NMS to process feature maps F2, feature maps P1 and feature maps P2, and outputs the final predicted rectangular box coordinates.

[0017] Furthermore, the CSP module includes two branches, branch one is a convolution layer, and branch two is composed of two convolution modules and a convolution layer connected in sequence, wherein the convolution module is composed of a convolution layer, a normalization layer, and an activation layer connected in sequence. Then, the outputs of the two branches are combined, and then processed by the normalization layer and the activation layer to obtain the output of the CSP module. The CSP module is used to first divide the input information into two parts, and then merge them through a cross-stage hierarchy, which can reduce the amount of calculation while maintaining the accuracy.

[0018] Furthermore, the image super-resolution model includes a feature extraction module, a feature learning module and a feature reconstruction module connected in sequence. Among them, the feature extraction module is composed of a convolutional neural network layer with a single convolution kernel of 3*3, which expands the number of feature channels and provides more feature information for feature learning of the subsequent modules; the feature learning module is composed of a trunk learning network and a gradient branch network, wherein the trunk learning network is composed of a plurality of enhanced information distillation modules EIMDB connected in series, and the outputs of all enhanced information distillation modules EIMDB are added by residual connection, so that low-dimensional and high-dimensional feature information are once again integrated to improve the learning ability of the model, and the gradient branch network is composed of a plurality of depth-separable convolutional layers connected in series, and the outputs of all depth-separable convolutional layers are added by residual connection. In addition, the gradient branch network uses the edge extraction operator Prewitt to extract the edge features of the input information, so that the gradient branch network can learn the edge features of the chalk characters in the blackboard writing area separately, so as to better optimize the reconstruction effect of super-resolution; the feature reconstruction module is composed of a pixel attention mechanism and a sub-pixel convolutional layer.

[0019] Furthermore, the structure of the enhanced information distillation module EIMDB includes two channel separation modules, a feature splicing layer, an enhanced spatial attention mechanism layer and a feature fusion layer connected in sequence, wherein the channel separation module is composed of a channel separation layer and a 3*3 convolution layer. First, in the channel separation module, the input feature channel is first divided into two parts of feature channels according to a ratio of 1:3 by the channel separation layer, which will produce two parts of feature channels, of which the part with a smaller proportion is retained and the other part is input into the 3*3 convolution layer. The retained part can be regarded as a refined feature channel, and the other part can be regarded as the output of the channel separation module after being processed by the 3*3 convolution layer. Then, all the refined feature channels and the output of the second channel separation module are passed to the feature splicing layer for channel splicing. The feature splicing layer and the enhanced spatial attention mechanism layer are connected in series, and the output of the enhanced spatial attention mechanism layer and the input end of the enhanced information distillation module EIMDB are residually connected, and fused through the feature fusion layer, and the output result is used as the output of the enhanced information distillation module EIMDB. The enhanced information distillation module EIMDB uses a progressive channel separation operation to fuse low-dimensional and high-dimensional feature information again, thereby improving the learning ability of the model.

[0020] Furthermore, the process of step S4 is as follows:

[0021] First, the bicubic upsampling interpolation method is used for the input low-resolution image. The bicubic upsampling interpolation basis function formula is:

[0022]

[0023] Among them, the parameters Indicates the distance between the pixel to be interpolated P(x,y) and its neighboring points in rows and columns. For the pixel to be interpolated P(x,y), x and y are the horizontal and vertical coordinates of the pixel to be interpolated, respectively. Take the 4*4 neighboring points P(x i ,y j ), where i, j = 0, 1, 2, 3, and the interpolation calculation is performed according to the following formula:

[0024]

[0025] In the formula, f(x,y) represents the pixel value to be interpolated, f(x i ,y j ) represents the 4*4 neighborhood point value near the pixel to be interpolated.

[0026] Then, the interpolated image I bicubic and reconstructed image I sr The mathematical expression for integration is as follows:

[0027]

[0028] In the formula, I out represents the final reconstruction effect image, and P represents the pixel point of the reconstruction effect image.

[0029] Compared with the prior art, the present invention has the following advantages and effects:

[0030] (1) The present invention proposes a super-resolution reconstruction method based on blackboard features in an online classroom scenario. Through the improved super-resolution model RVCSRN with gradient constraints, feature extraction is performed on chalk characters with obvious edge features in blackboard writing, which effectively improves the reconstruction effect of chalk characters with edge features in photos. The image super-resolution method is used to improve the resolution of the blackboard writing while effectively reducing the amount of data that needs to be transmitted during remote live broadcast, thereby reducing the requirements for network bandwidth.

[0031] (2) The present invention proposes a super-resolution reconstruction method based on blackboard features in an online classroom scenario. In the image preprocessing stage, the improved target detection model YOLOv5 is used to intelligently extract the blackboard area and eliminate non-blackboard areas such as walls and podiums, effectively reducing the size of the input image in the blackboard reconstruction process, thereby reducing the computational complexity and inference time of the super-resolution reconstruction method, and avoiding the interference of non-blackboard areas on the super-resolution reconstruction effect of the blackboard. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0033] Figure 1 It is a flowchart of a super-resolution reconstruction method based on blackboard features in an online classroom scenario disclosed by the present invention;

[0034] Figure 2 It is a working principle diagram of a super-resolution reconstruction method based on blackboard features in an online classroom scenario disclosed by the present invention;

[0035] Figure 3 is a schematic diagram of a classroom environment when a live broadcast device is shooting in an embodiment of the present invention;

[0036] Figure 4 Schematic diagram of different forms of the movable blackboard in the embodiment of the present invention;

[0037] Figure 5 It is a structural diagram of the improved target detection model YOLOv5 in an embodiment of the present invention;

[0038] Figure 6 RVCSRN is a structural diagram of an improved super-resolution model in an embodiment of the present invention;

[0039] Figure 7 It is a comparison diagram of the reconstruction effects of the improved super-resolution model RVCSRN in the embodiment of the present invention and other reconstruction methods. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] Example 1

[0042] As attached Figure 2 As shown, this embodiment provides a super-resolution reconstruction method based on blackboard features in an online classroom scenario, and the specific situation is as follows:

[0043] S1, as attached Figure 3 As shown, the camera is installed on the wall at the back of the classroom, collecting video information including the complete blackboard writing area in real time and transmitting the video information to the computer.

[0044] S2. The blackboard video information collected by S1 is processed by the interval frame extraction method to obtain the handwritten blackboard pictures, which are used as the blackboard high-resolution dataset for the online classroom scene. The bicubic downsampling interpolation method and video encoding and decoding operations are used to obtain the corresponding blackboard low-resolution dataset, which is then divided into the training dataset and test dataset for the image super-resolution task in a ratio of 7:3.

[0045] S3, preprocess the low-resolution blackboard image obtained in S2 for the target detection task. The preprocessing operation is to use LabelImg software to mark the edge of the blackboard area in the video frame, obtain the label file in xml format of the corresponding image, and the label file includes the border coordinates and category information of the blackboard area in the image, and divide the image and label into the training data set and the test data set of the target detection task in a ratio of 7:3.

[0046] S4. Considering that most of the blackboards in classrooms are movable, the position of the blackboard is not fixed. Figure 4 As shown in the figure, the traditional image calibration method is no longer applicable to extract the blackboard area, so the deep learning-based target detection model YOLOv5 is used to intelligently extract the blackboard area.

[0047] The improved target detection model YOLOv5 is trained using the target detection task training data set created in S3. This model mainly removes non-blackboard areas such as walls and podiums, reduces the area of ​​super-resolution reconstruction, and reduces interference with the super-resolution reconstruction effect, thereby improving reconstruction speed and reconstruction accuracy. The structure diagram of the improved target detection model YOLOv5 is shown in the attached figure. Figure 5 As shown, it includes a feature extraction module, a feature fusion module and a feature prediction module which are connected in sequence.

[0048] First, the feature extraction module is composed of a Focus module and three CSP modules connected in sequence. The Focus module contains four parallel slice layers and a convolution layer. After slicing the input image, it is spliced ​​in the channel dimension and convolution is performed. The outputs of the three CSP modules are recorded as feature maps M1, M2 and M3 in order. The CSP module contains two branches. Branch one is a convolution layer, and branch two is composed of two convolution modules and a convolution layer connected in sequence. The convolution module is composed of a convolution layer, a normalization layer and an activation layer connected in sequence. Then, the outputs of the two branches are spliced, and then processed by the normalization layer and the activation layer to obtain the output of the CSP module. The feature extraction module divides the data into 4 parts, each of which is equivalent to 2 times downsampling, and then splices it in the channel dimension, which can improve the receptive field of each point and minimize information loss.

[0049] Then, the feature fusion module is composed of a feature pyramid module FPN and a path aggregation module PAN. The feature pyramid module FPN uses an upsampling method to fuse information. The feature map M3 is first upsampled, and then fused with the feature map M2 to obtain the feature map F1. The feature map F1 is then upsampled and fused with the feature map M1 to obtain the feature map F2. The path aggregation module PAN uses a bottom-up method to fuse features. The feature map F2 first passes through a convolution layer with a step size of 2, and then is fused with the feature map F1 to obtain the feature map P1. The feature map P1 passes through a convolution layer with a step size of 2, and then is fused with the feature map M3 to obtain the feature map P2. The feature fusion module uses the feature pyramid module FPN to convey strong semantic features from top to bottom, and uses the path aggregation module PAN to convey strong positioning features from bottom to top, and performs feature fusion on different feature maps from different dimensions.

[0050] Finally, the feature prediction module uses non-maximum suppression NMS to process feature maps F2, feature maps P1 and feature maps P2, and outputs the final predicted rectangular box coordinates.

[0051] S5. According to the predicted rectangular frame coordinates obtained in step S4, the high-resolution and low-resolution image pairs obtained in step S2 are cropped at corresponding positions to obtain a blackboard area image, which are recorded as the real label data HR and input data LR of the super-resolution network respectively.

[0052] S6. Perform super-resolution reconstruction on the low-resolution blackboard area image LR. The super-resolution model uses the improved image super-resolution model RVCSRN with gradient constraints. The structure of the model is shown in the attached figure. Figure 6 As shown, it includes a feature extraction module, a feature learning module and a feature reconstruction module which are connected in sequence.

[0053] The feature extraction module consists of a single convolutional neural network layer with a 3*3 convolution kernel, which expands the number of feature channels and provides more feature information for feature learning in subsequent modules.

[0054] The feature learning module consists of a backbone learning network and a gradient branch network, wherein the backbone learning network consists of multiple enhanced information distillation modules EIMDB connected in series, and the outputs of all enhanced information distillation modules EIMDB are added using residual connections. The enhanced information distillation module EIMDB includes two channel separation modules, a feature splicing layer, an enhanced spatial attention mechanism layer, and a feature fusion layer connected in sequence, wherein the channel separation module consists of a channel separation layer and a 3*3 convolution layer. First, in the channel separation module, the input feature channel is first divided into two parts of feature channels according to a ratio of 1:3 by the channel separation layer, which will produce two parts of feature channels, of which the part with a smaller proportion is retained and the other part is input into the 3*3 convolution layer. The retained part can be regarded as a refined feature channel, and the other part can be regarded as the output of the channel separation module after being processed by the 3*3 convolution layer. Then, all the refined feature channels and the outputs of the second channel separation module are passed to the feature splicing layer for channel splicing. The feature splicing layer and the enhanced spatial attention mechanism layer are connected in series, and the output of the enhanced spatial attention mechanism layer is residually connected to the input of the enhanced information distillation module EIMDB, and fused through the feature fusion layer. The output result is used as the output of the enhanced information distillation module EIMDB. The enhanced information distillation module EIMDB uses a progressive channel separation operation to fuse low-dimensional and high-dimensional feature information again, improving the learning ability of the model.

[0055] The gradient branch network in the feature learning module is composed of multiple depthwise separable convolutional layers connected in series, and the outputs of all depthwise separable convolutional layers are added using residual connections. In addition, the gradient branch network uses the edge extraction operator Prewitt to extract the edge features of the input information, so that the gradient branch network can separately learn the edge features of the chalk characters in the blackboard area to better optimize the super-resolution reconstruction effect.

[0056] The feature reconstruction module consists of a pixel attention mechanism and a sub-pixel convolutional layer, which enhances the exchange of channel information during the feature reconstruction process, thereby improving the reconstruction performance of the model.

[0057] S7. The improved RVCSRN network was trained using the super-resolution dataset. The Adam optimization method and the initial learning rate of 0.001 were used for training, and the number of training epochs was 1000. The loss function used the mean absolute error loss function. When the loss function reached the set threshold, the training ended and the model parameters were selected as the final prediction model.

[0058] The image quality effect is evaluated using an evaluation criteria based on the error-sensitive index PSNR and the structural similarity index SSIM.

[0059] The calculation formula of the PSNR evaluation index is as follows:

[0060]

[0061] Among them, x and y are the model output image SR and the target image HR respectively, MaxValue is the maximum dynamic range that the image can take, and H and W are the height and width of the image.

[0062] The calculation formula of the SSIM evaluation index is as follows:

[0063]

[0064] Among them, x and y are the model output image SR and the target image HR, μ x is the mean of x, μ y is the mean of y, is the variance of x, is the variance of y, σ xy is the covariance of x and y, c1 and c2 are constants.

[0065] The PSNR and SSIM indicators will increase with the improvement of reconstruction quality. Finally, the blackboard image of the test set is input into the prediction model for super-resolution reconstruction, the corresponding image reconstruction quality performance indicators and model inference time are recorded, and the reconstruction effect image output by the super-resolution network is obtained.

[0066] S8, performing bicubic upsampling interpolation on the low-resolution image obtained in step S2 to obtain a Bicubic reconstruction effect image, wherein the bicubic upsampling interpolation basis function formula is:

[0067]

[0068] For the interpolated pixel point P(x,y), take its nearby 4*4 neighborhood points P(x i ,y j ), where i, j = 0, 1, 2, 3. The interpolation calculation is performed according to the following formula:

[0069]

[0070] In the formula, f(x,y) represents the pixel value to be interpolated, f(x i ,y j ) represents the 4*4 neighborhood point value near the pixel to be interpolated.

[0071] Then, the interpolated image I bicubic and the reconstructed image I output by the super-resolution model in step S7 sr The mathematical expression of integration is as follows:

[0072]

[0073] In the formula, I out Represents the final reconstruction effect image, and P represents the pixel point of the reconstruction effect image. Image I after bicubic upsampling interpolation bicubic and the reconstructed image I output by the super-resolution model in step S7 sr After integration through formula (e), the final output is the super-resolution reconstruction effect diagram based on blackboard features in the online classroom scenario I out .

[0074] Example 2

[0075] As shown in Table 1, the present embodiment provides a comparison between a method of first recognizing the blackboard writing area and then performing super-resolution reconstruction based on blackboard features (first recognition and then super-resolution) and a method of directly performing super-resolution reconstruction (direct super-resolution).

[0076] Table 1. Comparison of performance indicators between the recognition-first-super-resolution method and the direct super-resolution method

[0077]

[0078] The specific situation is as follows:

[0079] For the 1080P test image, when the area of ​​the blackboard writing area accounts for 1 / 2 of the entire test image, the method of first recognition and then super-resolution reduces the number of multiplications and additions and the time consumption compared with the direct super-resolution method; when the proportion decreases, the reduction in the number of multiplications and additions and the time consumption increases further. The experimental results verify the effectiveness of the method of first recognition and then super-resolution proposed in the present invention.

[0080] As shown in Table 2, this embodiment provides a comparison of the reconstruction performance of the super-resolution reconstruction model RVCSRN improved based on blackboard features and other image reconstruction methods in an online classroom scenario using a self-made blackboard dataset.

[0081] Table 2. Comparison of performance indicators of the improved super-resolution model RVCSRN and other reconstruction methods

[0082]

[0083] From Table 2, we can find that the improved super-resolution model RVCSRN outperforms other reconstruction methods in terms of image reconstruction performance indicators PSNR and SSIM. Figure 7 It can be seen that the image reconstruction renderings of the improved super-resolution model RVCSRN are clearer than those of other methods. The experimental results verify the effectiveness of the super-resolution reconstruction model RVCSRN based on the blackboard feature improvement proposed in the present invention.

[0084] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A super-resolution reconstruction method based on blackboard features in an online classroom scenario, characterized in that: The super-resolution reconstruction method comprises the following steps: S1, build blackboard high-resolution and low-resolution datasets for online classroom scenarios; S2. Construct a target detection model YOLOv5, and detect the input image through the target detection model YOLOv5, and use non-maximum suppression to output the four coordinate values ​​of the minimum circumscribed rectangle containing the blackboard writing area, so as to crop the blackboard writing area in the input image; the target detection model YOLOv5 includes a feature extraction module, a feature fusion module and a feature prediction module connected in sequence; the feature extraction module is composed of a Focus module and three CSP modules connected in sequence, wherein the Focus module includes four parallel slicing layers and a convolution layer, and the input image is sliced ​​and then spliced ​​in the channel dimension, and a convolution operation is performed, and the outputs of the three CSP modules are recorded as feature map M1, feature map M2 and feature map M3 in the order of front and back; the feature fusion module is composed of The feature pyramid module FPN and the path aggregation module PAN are composed of the feature pyramid module FPN, wherein the feature pyramid module FPN uses an upsampling method to perform feature fusion on the information, firstly upsamples the feature map M3, and then fuses it with the feature map M2 to obtain the feature map F1, and then the feature map F1 is upsampled and fused with the feature map M1 to obtain the feature map F2, and the path aggregation module PAN uses a bottom-up method to perform feature fusion, the feature map F2 first passes through a convolution layer with a step size of 2, and then performs feature fusion with the feature map F1 to obtain the feature map P1, and the feature map P1 passes through a convolution layer with a step size of 2, and then performs feature fusion with the feature map M3 to obtain the feature map P2; the feature prediction module uses non-maximum suppression NMS to process the feature maps F2, the feature maps P1 and the feature maps P2, and then outputs the final predicted rectangular frame coordinates; The CSP module includes two branches, branch one is a convolution layer, and branch two is composed of two convolution modules and a convolution layer connected in sequence, wherein the convolution module is composed of a convolution layer, a normalization layer and an activation layer connected in sequence, and then the outputs of the two branches are spliced, and then processed by the normalization layer and the activation layer to obtain the output of the CSP module; S3. Construct an image super-resolution model that introduces gradient constraints, and reconstruct the blackboard writing area with obvious edge features through the image super-resolution model to obtain a corresponding super-resolution reconstructed image; the image super-resolution model includes a feature extraction module, a feature learning module and a feature reconstruction module that are sequentially connected, wherein the feature extraction module is composed of a convolutional neural network layer with a single convolution kernel of 3*3, and the feature learning module is composed of a trunk learning network and a gradient branch network, wherein the trunk learning network is composed of a plurality of enhanced information distillation modules EIMDB connected in series, and then the outputs of all enhanced information distillation modules EIMDB are added by residual connection, the gradient branch network is composed of a plurality of depth-separable convolutional layers connected in series, and the outputs of all depth-separable convolutional layers are added by residual connection, and in addition, the gradient branch network uses the edge extraction operator Prewitt to extract edge features of input information; the feature reconstruction module is composed of a pixel attention mechanism and a sub-pixel convolutional layer; The enhanced information distillation module EIMDB includes a first channel separation module, a second channel separation module, a feature stitching layer, an enhanced spatial attention mechanism layer and a feature fusion layer connected in sequence, wherein the first and second channel separation modules are composed of a channel separation layer and a 3*3 convolution layer. First, in the first channel separation module, the input feature channel is first divided according to a 1:3 ratio through the channel separation layer to generate two parts of feature channels, wherein the part with a smaller proportion is retained, and the other part is input into the 3*3 convolution layer; the retained part is regarded as a refined feature channel, and the other part is regarded as the output of the channel separation module after being processed by the 3*3 convolution layer; then, all the refined feature channels and the output of the second channel separation module are transmitted to the feature stitching layer for channel stitching, the feature stitching layer and the enhanced spatial attention mechanism layer are connected in series, the output of the enhanced spatial attention mechanism layer and the input end of the enhanced information distillation module EIMDB are residually connected, and fused through the feature fusion layer, and the output result is used as the output of the enhanced information distillation module EIMDB; S4, performing bicubic upsampling interpolation on the low-resolution image input in step S2, and then integrating it with the reconstructed image of the blackboard area obtained in step S3, and outputting a super-resolution reconstruction result based on blackboard features in an online classroom scenario; the process of step S4 is as follows: First, the bicubic upsampling interpolation method is used for the input low-resolution image, where the bicubic upsampling interpolation basis function formula is: （1） Among them, the parameters Indicates the pixel to be interpolated The distance between the pixel's neighbors in rows and columns, to be interpolated , x and y are the horizontal and vertical coordinates of the pixel to be interpolated, and the 4*4 neighborhood points nearby are taken ,in , interpolation calculation is performed according to the following formula: （2） In the formula, Represents the pixel value to be interpolated. Represents the value of the 4*4 neighborhood points near the pixel to be interpolated; Then, the interpolated image and reconstruct the image The mathematical expression for integration is as follows: （3） In the formula, Represents the final reconstruction effect. Represents the pixel point of the reconstructed effect image.

2. According to claim 1, the super-resolution reconstruction method based on blackboard features in an online classroom scenario is characterized in that: The process of step S1 is as follows: The online classroom video is recorded by a camera, and then the video is framed at intervals to obtain a series of blackboard images. The series of blackboard images are used as the high-resolution blackboard dataset of the online classroom scene, and the bicubic downsampling interpolation method is used to obtain the corresponding low-resolution blackboard dataset, which is then divided into training dataset and test dataset in a ratio of 7:3.

Images