Whiteboard image enhancement method and system

Abstract

The present invention discloses a whiteboard image enhancement method and system, comprising: extracting features from an acquired whiteboard image to obtain shallow features and high-frequency features; performing high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance high-frequency information in the shallow features; and obtaining a whiteboard enhanced image based on the high-frequency enhanced features and the high-frequency features. This application can improve the image enhancement effect of whiteboard images.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement, and in particular to a whiteboard image enhancement method and system. Background Art

[0002] As a flexible medium for communication, whiteboard images have been widely used in education, offices, hospitals, and homes. However, the clarity of the content on the whiteboard image seriously affects the quality of communication.

[0003] In real-world scenarios, whiteboard images are affected by a variety of complex factors, such as ambient lighting, shadows, noise, blur, and light highlights. These factors can lead to degradation of whiteboard image quality and reduced readability.

[0004] Most existing whiteboard image enhancement methods focus on enhancing a single degraded scene. They are unable to effectively remove shadows, highlights, and noise from whiteboard images, and can also cause problems such as missing and blurred text. Therefore, it is particularly important to develop a whiteboard image enhancement method to improve the image enhancement effect of whiteboard images. Summary of the Invention

[0005] The present application provides a whiteboard image enhancement method and system to improve the image enhancement effect of whiteboard images.

[0006] In a first aspect, the present application provides a whiteboard image enhancement method, comprising:

[0007] Perform feature extraction on the acquired whiteboard image to obtain shallow features and high-frequency features;

[0008] Performing high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance high-frequency information in the shallow features;

[0009] A whiteboard enhanced image is obtained based on the high-frequency enhancement feature and the high-frequency feature.

[0010] The embodiment of the present application extracts shallow features and high-frequency features respectively, and adopts different processing methods for the shallow features and high-frequency features to effectively retain the shallow features and high-frequency information in the whiteboard image, thereby reducing the loss of high-frequency information caused by the existing whiteboard image enhancement method, and the inability to effectively remove shadows and other problems; by performing high-frequency enhancement processing on shallow features, the high-frequency information in the shallow features can be enhanced, and the problem of high-frequency information being continuously lost as the network reasoning depth deepens can be prevented; finally, a whiteboard enhanced image is obtained based on the high-frequency enhancement features and the high-frequency features, which not only reduces the loss of high-frequency information, but also effectively removes image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0011] Furthermore, before extracting features from the acquired whiteboard image, the method further includes:

[0012] An initial whiteboard image is acquired and simulation processing is performed on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, and resolution reduction processing.

[0013] In this way, by simulating the initial whiteboard image, the real whiteboard image and its degradation process can be simulated, and a large number of whiteboard images can be generated.

[0014] Furthermore, the feature extraction is performed on the acquired whiteboard image to obtain shallow features and high-frequency features, specifically:

[0015] Downsampling the whiteboard image to obtain a downsampled image, and performing feature extraction on the downsampled image to obtain the shallow features;

[0016] Feature extraction is performed on the whiteboard image to obtain high-frequency features in the whiteboard image.

[0017] In this way, by downsampling the whiteboard image, the network computational load can be reduced, and shallow features and high-frequency features containing high-frequency information such as handwriting can be extracted separately, making it convenient to subsequently adopt different processing methods for shallow features and high-frequency features, so as to effectively retain the shallow features and high-frequency information in the whiteboard image.

[0018] Furthermore, the high-frequency enhancement processing is performed on the shallow features to obtain high-frequency enhanced features, specifically:

[0019] Extracting shallow features to obtain shallow high-frequency features;

[0020] High-frequency enhancement processing is performed on the shallow high-frequency features to obtain high-frequency enhanced features.

[0021] In this way, by extracting high-frequency information from shallow features, the high-frequency information of shallow features can be effectively enhanced, which can prevent the problem of high-frequency information being lost as the depth of network reasoning deepens.

[0022] Furthermore, the whiteboard enhanced image is obtained based on the high-frequency enhancement feature and the high-frequency feature, specifically:

[0023] Extracting the high-frequency enhancement features to obtain deep high-frequency features;

[0024] Adding the shallow features to the deep high-frequency features, and performing feature extraction and upsampling on the addition results to obtain mixed enhanced features;

[0025] Fusing the hybrid enhancement feature with the high-frequency feature to obtain a fused enhancement feature;

[0026] The fused enhancement features are reconstructed to output a whiteboard enhanced image.

[0027] In this way, a whiteboard enhanced image is obtained based on the high-frequency enhancement features and the high-frequency features, which reduces the loss of high-frequency information and effectively removes image noise, shadows and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0028] In a second aspect, the present application provides a whiteboard image enhancement system, comprising: an extraction module, an enhancement module, and a fusion module;

[0029] The extraction module is used to extract features from the acquired whiteboard image to obtain shallow features and high-frequency features;

[0030] The enhancement module is configured to perform high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance high-frequency information in the shallow features;

[0031] The fusion module is used to obtain a whiteboard enhanced image based on the high-frequency enhancement feature and the high-frequency feature.

[0032] The embodiment of the present application extracts shallow features and high-frequency features respectively, and adopts different processing methods for the shallow features and high-frequency features to effectively retain the shallow features and high-frequency information in the whiteboard image, thereby reducing the loss of high-frequency information caused by the existing whiteboard image enhancement method, and the inability to effectively remove shadows and other problems; by performing high-frequency enhancement processing on shallow features, the high-frequency information existing in the shallow features can be enhanced, and the problem of high-frequency information being continuously lost as the network reasoning depth deepens can be prevented; finally, a whiteboard enhanced image is obtained based on the high-frequency enhancement features and the high-frequency features, which not only reduces the loss of high-frequency information, but also effectively removes image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0033] Furthermore, the whiteboard image enhancement system further includes: an acquisition module;

[0034] The acquisition module is used to acquire an initial whiteboard image and perform simulation processing on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, and resolution reduction processing.

[0035] Furthermore, the extraction module includes: a first extraction unit and a second extraction unit;

[0036] The first extraction unit is configured to perform downsampling processing on the whiteboard image to obtain a downsampled image, and perform feature extraction on the downsampled image to obtain the shallow features;

[0037] The second extraction unit performs feature extraction on the whiteboard image to obtain high-frequency features in the whiteboard image.

[0038] Furthermore, the enhancement module includes: a third extraction unit and a processing unit;

[0039] The third extraction unit is used to extract the shallow features to obtain shallow high-frequency features;

[0040] The processing unit is used to perform high-frequency enhancement processing on the shallow high-frequency features to obtain high-frequency enhanced features.

[0041] Furthermore, the fusion module includes: a fourth extraction unit, a fifth extraction unit, a fusion unit and an output unit;

[0042] The fourth extraction unit is used to extract the high-frequency enhancement features to obtain deep high-frequency features;

[0043] The fifth extraction unit is configured to add the shallow layer features to the deep layer high-frequency features, and perform feature extraction and upsampling on the addition result to obtain a mixed enhanced feature;

[0044] The fusion unit is used to fuse the mixed enhancement feature with the high-frequency feature to obtain a fused enhancement feature;

[0045] The output unit is used to reconstruct the fused enhancement features and output a whiteboard enhanced image. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flowchart of an embodiment of a whiteboard image enhancement method provided by the present application;

[0047] Figure 2 This is a flowchart of another embodiment of a whiteboard image enhancement method provided by the present application;

[0048] Figure 3 This is a schematic diagram of the structure of the high-frequency information enhancement unit provided by this application;

[0049] Figure 4 This is a structural diagram of an embodiment of a whiteboard image enhancement system provided by the present application. DETAILED DESCRIPTION

[0050] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0051] It should be understood that the step numbers used herein are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0052] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0053] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0054] The term "and / or" refers to and includes any and all possible combinations of one or more of the associated listed items.

[0055] As a medium for flexible communication, whiteboards have been widely used in education, offices, hospitals, and homes. The clarity of the content on the whiteboard image seriously affects the quality of communication. Based on this, this application proposes a whiteboard image enhancement method that can effectively improve the image enhancement effect of whiteboard images. By enhancing the whiteboard image, it can help improve the quality of whiteboard images, promote information transmission, and improve work efficiency in scenarios such as education, offices, hospitals, and homes.

[0056] Please refer to Figure 1 , Figure 1 This is a flow chart of an embodiment of a whiteboard image enhancement method provided by the present application, including steps S1 to S3:

[0057] Step S1: extract features from the acquired whiteboard image to obtain shallow features and high-frequency features;

[0058] Furthermore, before performing feature extraction on the acquired whiteboard image, it also includes: acquiring an initial whiteboard image, and performing simulation processing on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, resolution reduction processing and other simulation processing methods.

[0059] It should be noted that the initial whiteboard image refers to the original whiteboard image obtained by various methods such as image acquisition equipment or exporting from electronic whiteboard software. The image contains a whiteboard with background, shadows and writing information. Features rich in spatial information such as background information are shallow features, and features with large grayscale changes such as handwriting are high-frequency features.

[0060] Specifically, the text color processing step is: randomly selecting a scene from the data set of the initial whiteboard image, judging whether the pixel is background or text by calculating the average value of the three RGB channel pixels at each pixel position, and changing the pixel value judged as text. When the pixel value average value exceeds a certain value, the pixel is judged to be background, otherwise it is text, and the pixel value of the text is changed to the pixel value of the selected color to achieve text color transformation; the overall brightness processing step is: adjusting the pixel value of the image to simulate scenes in different lighting environments by adjusting the overall brightness of the image; the noise addition processing step is: adding noise such as Gaussian distribution or Poisson distribution noise to the image; the intensity blur processing step is: using a blur kernel to blur the image, where the blur kernel types include Gaussian blur, isotropic blur and anisotropic blur; the resolution reduction processing step is: reducing the image resolution through Bi cubic interpolation or Bi li near interpolation.

[0061] In this way, by simulating the initial whiteboard image, the real whiteboard image and its degradation process can be simulated, and a large number of whiteboard image data pairs can be generated.

[0062] After the acquired initial whiteboard image is simulated to obtain a whiteboard image, feature extraction can be performed on the whiteboard image to obtain shallow features and high-frequency features;

[0063] Specifically, downsampling the whiteboard image to obtain a downsampled image, and performing feature extraction on the downsampled image to obtain the shallow features;

[0064] Feature extraction is performed on the whiteboard image to obtain high-frequency features in the whiteboard image.

[0065] It should be noted that the whiteboard image may be downsampled by a downsampling module, wherein the downsampling module may downsample the whiteboard image by a factor of 2 or 4. This embodiment does not impose a limitation on the downsampling multiple.

[0066] It should be noted that shallow features are rich in spatial information, that is, intuitive information such as the position and shape of objects in the image, while high-frequency features refer to features with large grayscale changes such as handwriting in the image.

[0067] It should be noted that feature extraction can be achieved through convolutional layers, and the convolutional layers that extract shallow features and high-frequency features are different convolutional layers. Among them, shallow features are usually the output of the first one or two layers of the convolutional layer network, and the output of a convolutional layer network with three or more layers is the deep feature.

[0068] In this way, by downsampling the whiteboard image, the network computational load can be reduced, and shallow features and high-frequency features containing high-frequency information such as handwriting can be extracted separately, making it convenient to subsequently adopt different processing methods for shallow features and high-frequency features, so as to effectively retain the shallow features and high-frequency information in the whiteboard image.

[0069] Step S2: performing high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance high-frequency information in the shallow features;

[0070] Specifically, feature extraction is performed on the shallow features to obtain shallow high-frequency features; and high-frequency enhancement processing is performed on the shallow high-frequency features to obtain high-frequency enhancement features.

[0071] It should be noted that the shallow high-frequency features are used as input features and are sequentially passed through multiple convolutional layers and ReLU layers, alternating therewith. The shallow high-frequency information is then normalized by a Sigma d layer to obtain high-frequency information attention. The high-frequency information attention is then element-wise multiplied with the input features, i.e., the shallow high-frequency features, to obtain high-frequency enhanced features. Finally, the high-frequency enhanced features are used as input features, and the above operation is repeated multiple times to obtain the final high-frequency enhanced features. It should be noted that the number of repetitions is 2 or more.

[0072] It should be noted that each convolution layer and each ReLU layer in the alternating convolution layers and ReLU layers may be different, and this embodiment does not limit this.

[0073] In this way, by enhancing the high-frequency information in shallow features, the high-frequency information of shallow features can be effectively enhanced, which can prevent the problem of high-frequency information being lost as the depth of network reasoning deepens.

[0074] Step S3: obtaining a whiteboard enhanced image based on the high-frequency enhancement feature and the high-frequency feature.

[0075] Specifically, first, feature extraction is performed on the high-frequency enhancement feature to obtain a deep high-frequency feature; wherein the feature extraction of the high-frequency enhancement feature can be achieved through a convolutional layer;

[0076] Secondly, the shallow features described in S1 are added to the deep high-frequency features, and the added results are subjected to feature extraction and upsampling processing to obtain mixed enhanced features; it can be understood that the added results can be subjected to feature extraction through the convolution layer, and the downsampled image can be restored to the size of the original whiteboard image through upsampling processing, and the multiple of the upsampling processing corresponds to the multiple of the downsampling processing.

[0077] Next, the hybrid enhanced features are fused with the high-frequency features to obtain fused enhanced features. It can be understood that feature concatenation is achieved by stacking the hybrid enhanced features and the high-frequency features in the channel dimension, and feature fusion is achieved through a convolutional layer. It should be noted that the convolutional layer that extracts features from the enhanced features, the convolutional layer that extracts features from the summed results, and the convolutional layer that performs feature fusion can be different.

[0078] Finally, the fused enhancement features are reconstructed to output a whiteboard enhanced image. It can be understood that by reconstructing the fused enhancement features, the whiteboard enhanced image is directly obtained.

[0079] It should be noted that if the simulation process in step S1 adopts a step of reducing the resolution, then it is necessary to perform upsampling or super-resolution reconstruction on the whiteboard enhanced image to restore the resolution of the original whiteboard image.

[0080] In this way, a whiteboard enhanced image is obtained based on the high-frequency enhancement feature and the high-frequency feature, which reduces the loss of high-frequency information while effectively removing image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0081] The embodiment of the present application extracts shallow features and high-frequency features respectively, and adopts different processing methods for the shallow features and high-frequency features to effectively retain the shallow features and high-frequency information in the whiteboard image, thereby reducing the loss of high-frequency information caused by the existing whiteboard image enhancement method, and the inability to effectively remove shadows and other problems; by performing high-frequency enhancement processing on shallow features, the high-frequency information in the shallow features can be enhanced, and the problem of high-frequency information being continuously lost as the network reasoning depth deepens can be prevented; finally, a whiteboard enhanced image is obtained based on the high-frequency enhancement features and the high-frequency features, which not only reduces the loss of high-frequency information, but also effectively removes image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0082] This application provides Figure 2 Easy to understand, Figure 2 This is a flowchart of another embodiment of a whiteboard image enhancement method provided by the present application;

[0083] First, the whiteboard image is directly input into the first convolutional layer to extract high-frequency features from the whiteboard image. In addition, the whiteboard image is input into the downsampling module for downsampling processing to obtain a downsampled image, and the downsampled image is input into the second convolutional layer to extract shallow features from the downsampled image.

[0084] Secondly, the shallow features are input into a high-frequency information enhancement module for high-frequency enhancement processing to obtain high-frequency enhanced features, wherein the high-frequency information enhancement module may include a third convolutional layer and a high-frequency information enhancement unit, and the high-frequency information in the shallow features may be enhanced by the high-frequency information enhancement module to obtain high-frequency enhanced features;

[0085] The high-frequency information enhancement unit may include an eighth convolutional layer, a first ReLU layer, a ninth convolutional layer, a second ReLU layer, a tenth convolutional layer, a third ReLU layer, an eleventh convolutional layer and a Sigmoi d layer. By alternating multiple convolutional layers and ReLU layers, high-frequency information in shallow features can be extracted, and high-frequency information attention is obtained through the Sigmoi d layer; then the high-frequency information attention is element-wise multiplied with the shallow features to obtain high-frequency enhanced features. The schematic diagram of the high-frequency information enhancement unit is shown as follows Figure 3 shown.

[0086] It should be noted that there may be multiple high-frequency information enhancement modules, the eighth convolutional layer, the ninth convolutional layer, the tenth convolutional layer and the eleventh convolutional layer may be different convolutional layers; the first ReLU layer, the second ReLU layer and the third ReLU layer may be different ReLU layers.

[0087] Again, the high-frequency enhancement features are input into the fourth convolutional layer for feature extraction to obtain deep high-frequency features; and the shallow features are added to the deep high-frequency features, and the addition results are input into the fifth convolutional layer, the upsampling module and the sixth convolutional layer in sequence to restore the size of the original whiteboard image and obtain mixed enhancement features.

[0088] It should be noted that the fourth convolutional layer, the fifth convolutional layer, and the sixth convolutional layer may be different convolutional layers.

[0089] Feature splicing is achieved by stacking the hybrid enhancement features and the high-frequency features in the channel dimension, and feature fusion is achieved through the seventh convolutional layer to obtain fused enhancement features; finally, based on the fused enhancement features and the whiteboard image, a whiteboard enhanced image is output.

[0090] It should be noted that if the simulation process adopts a step of reducing the resolution, then it is necessary to perform upsampling processing on the whiteboard enhanced image to restore the resolution of the original whiteboard image.

[0091] Please refer to Figure 4 , Figure 4 This is a structural diagram of an embodiment of a whiteboard image enhancement system provided by the present application, comprising an extraction module 01, an enhancement module 02, and a fusion module 03;

[0092] The extraction module 01 is used to extract features from the acquired whiteboard image to obtain shallow features and high-frequency features;

[0093] Furthermore, the whiteboard image enhancement system also includes: the acquisition module, which is used to acquire the initial whiteboard image and perform simulation processing on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, resolution reduction processing and other simulation processing methods.

[0094] It should be noted that the initial whiteboard image refers to the original image captured by the image acquisition device. The image contains a whiteboard with background, shadows and written information. Features rich in spatial information such as background information are shallow features, and features with large grayscale changes such as handwriting are high-frequency features.

[0095] Specifically, the text color processing step is: randomly selecting a scene from the data set of the initial whiteboard image, judging whether the pixel is background or text by calculating the average value of the three RGB channel pixels at each pixel position, and changing the pixel value judged as text. When the pixel value average value exceeds a certain value, the pixel is judged to be background, otherwise it is text, and the pixel value of the text is changed to the pixel value of the selected color to achieve text color transformation; the overall brightness processing step is: adjusting the pixel value of the image, and then adjusting the overall brightness of the image to simulate scenes in different lighting environments; the noise addition processing step is: adding noise such as Gaussian distribution or Poisson distribution noise to the image; the intensity blur processing step is: using a blur kernel to blur the image, where the blur kernel types include Gaussian blur, isotropic blur and anisotropic blur; the resolution reduction processing step is: reducing the image resolution through Bicubic interpolation or Bi li near interpolation.

[0096] In this way, by simulating the initial whiteboard image, the real whiteboard image and its degradation process can be simulated, and a large number of whiteboard image data pairs can be generated.

[0097] After the acquired initial whiteboard image is simulated to obtain a whiteboard image, feature extraction can be performed on the whiteboard image to obtain shallow features and high-frequency features; specifically, the extraction module 01 includes: a first extraction unit and a second extraction unit; the first extraction unit is used to downsample the whiteboard image to obtain a downsampled image, and perform feature extraction on the downsampled image to obtain the shallow features; the second extraction unit is used to perform feature extraction on the whiteboard image to obtain high-frequency features in the whiteboard image.

[0098] It should be noted that the whiteboard image may be downsampled by a downsampling module, wherein the downsampling module may downsample the whiteboard image by a factor of 2 or 4. This embodiment does not impose a limitation on the downsampling multiple.

[0099] It should be noted that shallow features are rich in spatial information, that is, intuitive information such as the position and shape of objects in the image, while high-frequency features refer to features with large grayscale changes such as handwriting in the image.

[0100] It should be noted that feature extraction can be achieved through convolutional layers, and the convolutional layers that extract shallow features and high-frequency features are different convolutional layers. Among them, shallow features are usually the output of the first one or two layers of the convolutional layer network, and the output of a convolutional layer network with three or more layers is the deep feature.

[0101] In this way, by downsampling the whiteboard image, the network computational load can be reduced, and shallow features containing more pixel information and high-frequency features containing high-frequency information such as handwriting can be extracted separately, making it convenient to subsequently adopt different processing methods for shallow features and high-frequency features, and effectively retaining shallow features and high-frequency information in the whiteboard image.

[0102] The enhancement module 02 is configured to perform high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance high-frequency information in the shallow features;

[0103] The enhancement module includes: a third extraction unit and a processing unit;

[0104] The third extraction unit is used to extract the shallow features to obtain shallow high-frequency features; the processing unit is used to perform high-frequency enhancement processing on the shallow high-frequency features to obtain high-frequency enhanced features;

[0105] It should be noted that the shallow high-frequency features are used as input features and are sequentially passed through multiple convolutional layers and ReLU layers, alternating therewith. The shallow high-frequency information is then normalized by a Sigmoid layer to obtain high-frequency information attention. The high-frequency information attention is then element-wise multiplied with the input features, i.e., the shallow high-frequency features, to obtain high-frequency enhanced features. Finally, the high-frequency enhanced features are used as input features, and the above operation is repeated multiple times to obtain the final high-frequency enhanced features. It should be noted that the number of repetitions is 2 or more.

[0106] It should be noted that each convolution layer and each ReLU layer in the alternating convolution layers and ReLU layers may be different, and this embodiment is not limited to this feature extraction method.

[0107] In this way, by extracting high-frequency information from shallow features, the high-frequency information of shallow features can be effectively enhanced, which can prevent the problem of high-frequency information being lost as the depth of network reasoning deepens.

[0108] The fusion module 03 is configured to obtain a whiteboard enhanced image based on the high-frequency enhancement feature and the high-frequency feature.

[0109] Specifically, the fusion module includes: a fourth extraction unit, a fifth extraction unit, a fusion unit and an output unit;

[0110] The fourth extraction unit is used to extract the high-frequency enhancement features to obtain deep high-frequency features; wherein the feature extraction of the high-frequency enhancement features can be achieved through a convolution layer;

[0111] The fifth extraction unit is used to add the shallow features and the deep high-frequency features, and perform feature extraction and upsampling on the addition result to obtain a mixed enhanced feature; it can be understood that the addition result can be feature extracted through a convolution layer, and the downsampled image can be restored to the size of the original whiteboard image through upsampling, and the upsampling multiple corresponds to the downsampling multiple.

[0112] The fusion unit is configured to fuse the hybrid enhancement features with the high-frequency features to obtain fused enhancement features. It is understood that feature concatenation is achieved by stacking the hybrid enhancement features and the high-frequency features in the channel dimension, and feature fusion is achieved through a convolutional layer. It should be noted that the convolutional layer for extracting the enhancement features, the convolutional layer for extracting the features from the summed results, and the convolutional layer for performing feature fusion may be different.

[0113] The output unit is configured to reconstruct the fused enhancement features and output a whiteboard enhanced image. It is understandable that the whiteboard enhanced image can be directly obtained by reconstructing the fused enhancement features.

[0114] It should be noted that if the simulation process in the acquisition module adopts a step of reducing the resolution, then it is necessary to perform upsampling or super-resolution reconstruction on the whiteboard enhanced image to restore the resolution of the original whiteboard image.

[0115] In this way, a whiteboard enhanced image is obtained based on the high-frequency enhancement feature and the high-frequency feature, which reduces the loss of high-frequency information while effectively removing image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0116] The embodiment of the present application extracts shallow features and high-frequency features respectively, and adopts different processing methods for the shallow features and high-frequency features to effectively retain the shallow features and high-frequency information in the whiteboard image, thereby reducing the loss of high-frequency information caused by the existing whiteboard image enhancement method, and the inability to effectively remove shadows and other problems; by performing high-frequency enhancement processing on shallow features, the high-frequency information existing in the shallow features can be enhanced, and the problem of high-frequency information being continuously lost as the network reasoning depth deepens can be prevented; finally, a whiteboard enhanced image is obtained based on the high-frequency enhancement features and the high-frequency features, which not only reduces the loss of high-frequency information, but also effectively removes image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0117] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention by those skilled in the art should be included within the scope of protection of the present invention.

Claims

1. A whiteboard image enhancement method, characterized in that: include: Perform feature extraction on the acquired whiteboard image to obtain shallow features and high-frequency features; Performing feature extraction on the shallow features to obtain shallow high-frequency features, performing high-frequency enhancement processing on the shallow high-frequency features to obtain high-frequency enhanced features, wherein the shallow high-frequency features are alternately set through multiple convolutional layers and ReLU layers in sequence to obtain shallow high-frequency information, normalizing the shallow high-frequency information through a Sigmoid layer to obtain high-frequency information attention, and performing element-by-element multiplication of the high-frequency information attention and the shallow high-frequency features to obtain the high-frequency enhanced features; Based on the high-frequency enhancement features and the high-frequency features, a whiteboard enhanced image is obtained, specifically: feature extraction is performed on the high-frequency enhancement features to obtain deep high-frequency features; the shallow features are added to the deep high-frequency features, and feature extraction and upsampling are performed on the addition results to obtain mixed enhancement features; the mixed enhancement features are feature fused with the high-frequency features to obtain fused enhancement features; the fused enhancement features are reconstructed to output a whiteboard enhanced image.

2. The whiteboard image enhancement method according to claim 1, characterized in that: Before feature extraction is performed on the acquired whiteboard image, the following steps are also included: An initial whiteboard image is acquired and simulation processing is performed on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, and resolution reduction processing.

3. The whiteboard image enhancement method according to claim 1, characterized in that: The feature extraction is performed on the acquired whiteboard image to obtain shallow features and high-frequency features, specifically: Downsampling the whiteboard image to obtain a downsampled image, and performing feature extraction on the downsampled image to obtain the shallow features; Feature extraction is performed on the whiteboard image to obtain high-frequency features in the whiteboard image.

4. A whiteboard image enhancement system, characterized in that: include: Extraction module, enhancement module and fusion module; The extraction module is used to extract features from the acquired whiteboard image to obtain shallow features and high-frequency features; The enhancement module is used to extract features from the shallow features to obtain shallow high-frequency features, perform high-frequency enhancement processing on the shallow high-frequency features to obtain high-frequency enhanced features, wherein the shallow high-frequency features are alternately passed through multiple convolutional layers and ReLU layers to obtain shallow high-frequency information, normalize the shallow high-frequency information through a Sigmoid layer to obtain high-frequency information attention, and perform element-by-element multiplication of the high-frequency information attention and the shallow high-frequency features to obtain the high-frequency enhanced features; The fusion module is used to obtain a whiteboard enhanced image based on the high-frequency enhancement features and the high-frequency features, specifically by: extracting features from the high-frequency enhancement features to obtain deep high-frequency features; adding the shallow features to the deep high-frequency features, and performing feature extraction and upsampling on the addition results to obtain mixed enhancement features; fusing the mixed enhancement features with the high-frequency features to obtain fused enhancement features; and reconstructing the fused enhancement features to output a whiteboard enhanced image.

5. The whiteboard image enhancement system according to claim 4, characterized in that: Also includes: Get module; The acquisition module is used to acquire an initial whiteboard image and perform simulation processing on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, and resolution reduction processing.

6. The whiteboard image enhancement system according to claim 4, characterized in that: The extraction module includes: a first extraction unit and a second extraction unit; The first extraction unit is configured to perform downsampling processing on the whiteboard image to obtain a downsampled image, and perform feature extraction on the downsampled image to obtain the shallow features; The second extraction unit performs feature extraction on the whiteboard image to obtain high-frequency features in the whiteboard image.

Images