Whiteboard image enhancement method and system

By performing feature extraction and high-frequency enhancement processing on whiteboard images, combining shallow and high-frequency feature fusion, the problems of poor shadow, highlighting and noise processing in the existing technology are solved, and the efficient enhancement effect of whiteboard images is achieved and the improvement of the improvement of whiteboard images is improved. Image clarity and readability.

CN118967540AActive Publication Date: 2024-11-15YEALINK (XIAMEN) NETWORK TECHNOLOGY CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN202411028166.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-15
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

Existing whiteboard image enhancement methods cannot effectively remove shadows, highlights and noise, resulting in degradation of image quality and reduced readability of content, and problems such as high-frequency information loss and word loss.

Method used

By extracting features of the whiteboard images, obtaining shallow features and high-frequency features respectively, and performing high-frequency enhancement processing on shallow features, combining high-frequency enhancement features and high-frequency features to generate whiteboard enhanced images and reduce high-frequency information Lost and remove noise and highlights.

Benefits of technology

It effectively improves the image enhancement effect of whiteboard images, reduces the loss of high-frequency information, prevents the loss of high-frequency information as the network inference deepens, and removes image noise and highlights, improving the clarity and readability of the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967540A_ABST
    Figure CN118967540A_ABST
Patent Text Reader

Abstract

The invention discloses a whiteboard image enhancement method and system, and the method comprises the steps: carrying out the feature extraction of an obtained whiteboard image, and obtaining a shallow feature and a high-frequency feature; performing high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, the high-frequency enhancement processing being capable of enhancing high-frequency information in the shallow features; and obtaining a whiteboard enhanced image based on the high-frequency enhanced feature and the high-frequency feature. According to the invention, the image enhancement effect of the whiteboard image can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image enhancement, and in particular to a whiteboard image enhancement method and system. Background Art

[0002] As a medium for flexible communication, whiteboard images have been widely used in education, office, hospital, and home. However, the clarity of the content on the whiteboard image seriously affects the quality of communication.

[0003] In real-world scenarios, whiteboard images are affected by a variety of complex factors, such as ambient lighting, shadows, noise, blur, light highlights, etc. These factors can lead to degradation of whiteboard image quality and reduced content readability.

[0004] Most existing whiteboard image enhancement methods focus on enhancing a single degraded scene, and cannot effectively remove shadows, highlights, and noise in whiteboard images. In addition, there are problems such as handwriting loss and handwriting blur. Therefore, it is particularly important to study a whiteboard image enhancement method to improve the image enhancement effect of whiteboard images. Summary of the invention

[0005] The present application provides a whiteboard image enhancement method and system to improve the image enhancement effect of a whiteboard image.

[0006] In a first aspect, the present application provides a whiteboard image enhancement method, comprising:

[0007] Perform feature extraction on the acquired whiteboard image to obtain shallow features and high-frequency features respectively;

[0008] Performing high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance high-frequency information in the shallow features;

[0009] Based on the high-frequency enhancement feature and the high-frequency feature, a whiteboard enhanced image is obtained.

[0010] The embodiment of the present application extracts shallow features and high-frequency features respectively, and adopts different processing methods for the shallow features and high-frequency features to effectively retain the shallow features and high-frequency information in the whiteboard image, thereby reducing the loss of high-frequency information caused by the existing whiteboard image enhancement method, and the inability to effectively remove shadows and other problems; by performing high-frequency enhancement processing on shallow features, the high-frequency information in the shallow features can be enhanced, and the problem of high-frequency information being continuously lost as the network reasoning depth deepens can be prevented; finally, a whiteboard enhanced image is obtained based on the high-frequency enhancement features and the high-frequency features, which reduces the loss of high-frequency information while effectively removing image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0011] Furthermore, before extracting features from the acquired whiteboard image, the method further includes:

[0012] An initial whiteboard image is acquired, and simulation processing is performed on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, and resolution reduction processing.

[0013] In this way, by simulating the initial whiteboard image, the real whiteboard image and its degradation process can be simulated, and a large number of whiteboard images can be generated.

[0014] Furthermore, the feature extraction is performed on the acquired whiteboard image to obtain shallow features and high-frequency features, respectively, specifically:

[0015] Downsampling the whiteboard image to obtain a downsampled image, and extracting features from the downsampled image to obtain the shallow features;

[0016] Feature extraction is performed on the whiteboard image to obtain high-frequency features in the whiteboard image.

[0017] In this way, by downsampling the whiteboard image, the network calculation amount can be reduced, and shallow features and high-frequency features containing high-frequency information such as handwriting can be extracted separately, which facilitates the subsequent different processing methods for shallow features and high-frequency features, so as to effectively retain the shallow features and high-frequency information in the whiteboard image.

[0018] Furthermore, the high-frequency enhancement processing is performed on the shallow features to obtain high-frequency enhanced features, specifically:

[0019] Extracting the shallow features to obtain shallow high-frequency features;

[0020] The shallow high-frequency features are subjected to high-frequency enhancement processing to obtain high-frequency enhanced features.

[0021] In this way, by extracting high-frequency information in shallow features, the high-frequency information of shallow features can be effectively enhanced, which can prevent the problem of high-frequency information being lost as the depth of network reasoning increases.

[0022] Furthermore, the whiteboard enhanced image is obtained based on the high-frequency enhancement feature and the high-frequency feature, specifically:

[0023] Extracting the high-frequency enhancement features to obtain deep high-frequency features;

[0024] Adding the shallow layer features to the deep layer high frequency features, and performing feature extraction and upsampling processing on the addition result to obtain mixed enhanced features;

[0025] Fusing the mixed enhancement feature with the high-frequency feature to obtain a fused enhancement feature;

[0026] The fused enhanced features are reconstructed to output a whiteboard enhanced image.

[0027] In this way, a whiteboard enhanced image is obtained based on the high-frequency enhancement feature and the high-frequency feature, which reduces the loss of high-frequency information and can effectively remove image noise, shadows and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0028] In a second aspect, the present application provides a whiteboard image enhancement system, including: an extraction module, an enhancement module and a fusion module;

[0029] The extraction module is used to extract features from the acquired whiteboard image to obtain shallow features and high-frequency features respectively;

[0030] The enhancement module is used to perform high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance the high-frequency information in the shallow features;

[0031] The fusion module is used to obtain a whiteboard enhanced image based on the high-frequency enhancement feature and the high-frequency feature.

[0032] The embodiment of the present application extracts shallow features and high-frequency features respectively, and adopts different processing methods for the shallow features and high-frequency features to effectively retain the shallow features and high-frequency information in the whiteboard image, thereby reducing the loss of high-frequency information caused by the existing whiteboard image enhancement method, and the inability to effectively remove shadows and other problems; by performing high-frequency enhancement processing on the shallow features, the high-frequency information existing in the shallow features can be enhanced, and the problem of high-frequency information being continuously lost as the network reasoning depth deepens can be prevented; finally, a whiteboard enhanced image is obtained based on the high-frequency enhancement features and the high-frequency features, which reduces the loss of high-frequency information while effectively removing image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0033] Furthermore, the whiteboard image enhancement system further includes: an acquisition module;

[0034] The acquisition module is used to acquire an initial whiteboard image and perform simulation processing on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, and resolution reduction processing.

[0035] Further, the extraction module includes: a first extraction unit and a second extraction unit;

[0036] The first extraction unit is used to perform downsampling processing on the whiteboard image to obtain a downsampled image, and perform feature extraction on the downsampled image to obtain the shallow feature;

[0037] The second extraction unit performs feature extraction on the whiteboard image to obtain high-frequency features in the whiteboard image.

[0038] Further, the enhancement module includes: a third extraction unit and a processing unit;

[0039] The third extraction unit is used to extract the shallow features to obtain shallow high-frequency features;

[0040] The processing unit is used to perform high-frequency enhancement processing on the shallow high-frequency features to obtain high-frequency enhanced features.

[0041] Further, the fusion module includes: a fourth extraction unit, a fifth extraction unit, a fusion unit and an output unit;

[0042] The fourth extraction unit is used to extract the high-frequency enhancement features to obtain deep high-frequency features;

[0043] The fifth extraction unit is used to add the shallow layer features and the deep layer high-frequency features, and perform feature extraction and upsampling processing on the addition result to obtain a mixed enhanced feature;

[0044] The fusion unit is used to fuse the mixed enhancement feature with the high-frequency feature to obtain a fused enhancement feature;

[0045] The output unit is used to reconstruct the fused enhancement features and output a whiteboard enhanced image. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flowchart of an embodiment of a whiteboard image enhancement method provided by the present application;

[0047] Figure 2 is a flow chart of another embodiment of a whiteboard image enhancement method provided by the present application;

[0048] Figure 3 is a structural schematic diagram of a high-frequency information enhancement unit provided by the present application;

[0049] Figure 4 It is a structural schematic diagram of an embodiment of a whiteboard image enhancement system provided by the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] It should be understood that the step numbers used in this article are only for the convenience of description and are not intended to limit the order in which the steps are executed.

[0052] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0053] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0054] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0055] As a medium for flexible communication, whiteboards have been widely used in education, office, hospital, and home. The clarity of the content on the whiteboard image seriously affects the quality of communication. Based on this, the present application proposes a whiteboard image enhancement method, which can effectively improve the image enhancement effect of the whiteboard image. By enhancing the whiteboard image, it can help improve the quality of the whiteboard image, promote information transmission, and improve work efficiency in scenarios such as education, office, hospital, and home.

[0056] Please refer to Figure 1 , Figure 1 This is a flow chart of an embodiment of a whiteboard image enhancement method provided by the present application, including steps S1 to S3:

[0057] Step S1, extracting features from the acquired whiteboard image to obtain shallow features and high-frequency features;

[0058] Furthermore, before extracting features from the acquired whiteboard image, it also includes: acquiring an initial whiteboard image, and performing simulation processing on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise increase processing, intensity blur processing, resolution reduction processing, and other simulation processing methods.

[0059] It should be noted that the initial whiteboard image refers to the original whiteboard image obtained by various means such as image acquisition equipment or electronic whiteboard software export. The image contains a whiteboard with background, shadows and writing information. Features rich in spatial information such as background information are shallow features, and features with large grayscale changes such as handwriting are high-frequency features.

[0060] Specifically, the text color processing step is: randomly selecting a scene from the data set of the initial whiteboard image, judging whether the pixel is background or text by calculating the average value of the pixels of the three channels RGB at each pixel position, and changing the pixel value judged as text. When the average value of the pixel value exceeds a certain value, the pixel is judged as background, otherwise it is text, and the pixel value of the text is changed to the pixel value of the selected color to achieve text color change; the overall brightness processing step is: adjusting the pixel value of the image, and then adjusting the overall brightness of the image to simulate scenes in different lighting environments; the noise adding processing step is: adding noise such as Gaussian distribution or Poisson distribution noise to the image; the intensity blur processing step is: using a blur kernel to blur the image, wherein the blur kernel types include Gaussian blur, isotropic blur and anisotropic blur; the resolution reduction processing step is: reducing the image resolution by Bi cub ic interpolation or Bi li near interpolation.

[0061] In this way, by simulating the initial whiteboard image, the real whiteboard image and its degradation process can be simulated, and a large number of whiteboard image data pairs can be generated.

[0062] After the acquired initial whiteboard image is simulated to obtain a whiteboard image, feature extraction can be performed on the whiteboard image to obtain shallow features and high-frequency features;

[0063] Specifically, downsampling the whiteboard image to obtain a downsampled image, and extracting features from the downsampled image to obtain the shallow features;

[0064] Feature extraction is performed on the whiteboard image to obtain high-frequency features in the whiteboard image.

[0065] It should be noted that the whiteboard image may be downsampled by a downsampling module, wherein the downsampling module may downsample the whiteboard image by 2 or 4 times, and this embodiment does not limit the downsampling multiple.

[0066] It should be noted that shallow features are rich in spatial information, that is, intuitive information such as the position and shape of objects in the image, while high-frequency features refer to features with large grayscale changes such as handwriting in the image.

[0067] It should be noted that feature extraction can be achieved through convolutional layers, and the convolutional layers for extracting shallow features and high-frequency features are different convolutional layers. The shallow features are usually the output of the previous layer or two layers of the convolutional layer network, and the output of the convolutional layer network with three or more layers is the deep feature.

[0068] In this way, by downsampling the whiteboard image, the network calculation amount can be reduced, and shallow features and high-frequency features containing high-frequency information such as handwriting can be extracted separately, which facilitates the subsequent different processing methods for shallow features and high-frequency features, so as to effectively retain the shallow features and high-frequency information in the whiteboard image.

[0069] Step S2, performing high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance the high-frequency information in the shallow features;

[0070] Specifically, feature extraction is performed on the shallow features to obtain shallow high-frequency features; and high-frequency enhancement processing is performed on the shallow high-frequency features to obtain high-frequency enhancement features.

[0071] It should be noted that the shallow high-frequency features are used as input features, and are alternately set through multiple convolutional layers and ReLU layers in sequence, and the shallow high-frequency information is normalized through the Sigma layer to obtain high-frequency information attention. The high-frequency information attention is multiplied element by element with the input feature, i.e., the shallow high-frequency feature, to obtain the high-frequency enhancement feature. Finally, the high-frequency enhancement feature is used as the input feature, and the above operation is repeated multiple times to obtain the final high-frequency enhancement feature. It should be noted that the number of repetitions is 2 times or more.

[0072] It should be noted that each convolution layer and each ReLU layer in the alternating convolution layers and ReLU layers may be different, and this embodiment does not limit this.

[0073] In this way, by enhancing the high-frequency information in shallow features, the high-frequency information of shallow features can be effectively enhanced, which can prevent the problem of high-frequency information being lost as the depth of network reasoning deepens.

[0074] Step S3: obtaining a whiteboard enhanced image based on the high-frequency enhancement feature and the high-frequency feature.

[0075] Specifically, first, feature extraction is performed on the high-frequency enhancement feature to obtain a deep high-frequency feature; wherein the feature extraction of the high-frequency enhancement feature can be achieved through a convolutional layer;

[0076] Secondly, the shallow features described in S1 are added to the deep high-frequency features, and the added results are subjected to feature extraction and upsampling processing to obtain mixed enhanced features; it can be understood that the added results can be subjected to feature extraction through the convolution layer, and the downsampled image can be restored to the size of the original whiteboard image through upsampling processing, and the multiple of the upsampling processing corresponds to the multiple of the downsampling processing.

[0077] Again, the hybrid enhanced feature is fused with the high-frequency feature to obtain a fused enhanced feature; it can be understood that feature concatenation is achieved by stacking the hybrid enhanced feature and the high-frequency feature in the channel dimension, and feature fusion is achieved through the convolution layer. It should be noted that the convolution layer for extracting the enhanced feature, the convolution layer for extracting the feature of the addition result, and the convolution layer for performing feature fusion can be different.

[0078] Finally, the fused enhancement features are reconstructed to output a whiteboard enhanced image. It can be understood that by reconstructing the fused enhancement features, the whiteboard enhanced image is directly obtained.

[0079] It should be noted that if the simulation process in step S1 adopts a step of reducing the resolution, then it is necessary to perform upsampling or super-resolution reconstruction on the whiteboard enhanced image to restore the resolution of the initial whiteboard image.

[0080] In this way, a whiteboard enhanced image is obtained based on the high-frequency enhancement feature and the high-frequency feature, which reduces the loss of high-frequency information and can effectively remove image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0081] The embodiment of the present application extracts shallow features and high-frequency features respectively, and adopts different processing methods for the shallow features and high-frequency features to effectively retain the shallow features and high-frequency information in the whiteboard image, thereby reducing the loss of high-frequency information caused by the existing whiteboard image enhancement method, and the inability to effectively remove shadows and other problems; by performing high-frequency enhancement processing on shallow features, the high-frequency information in the shallow features can be enhanced, and the problem of high-frequency information being continuously lost as the network reasoning depth deepens can be prevented; finally, a whiteboard enhanced image is obtained based on the high-frequency enhancement features and the high-frequency features, which reduces the loss of high-frequency information while effectively removing image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0082] This application provides Figure 2 Easy to understand, Figure 2 is a flow chart of another embodiment of a whiteboard image enhancement method provided by the present application;

[0083] First, the whiteboard image is directly input into the first convolutional layer to extract the high-frequency features in the whiteboard image. In addition, the whiteboard image is input into the downsampling module for downsampling processing to obtain a downsampled image, and the downsampled image is input into the second convolutional layer to extract the shallow features in the downsampled image.

[0084] Secondly, the shallow features are input into a high-frequency information enhancement module for high-frequency enhancement processing to obtain high-frequency enhanced features, wherein the high-frequency information enhancement module may include a third convolutional layer and a high-frequency information enhancement unit, and the high-frequency information in the shallow features may be enhanced by the high-frequency information enhancement module to obtain high-frequency enhanced features;

[0085] The high-frequency information enhancement unit may include an eighth convolutional layer, a first ReLU layer, a ninth convolutional layer, a second ReLU layer, a tenth convolutional layer, a third ReLU layer, an eleventh convolutional layer and a Sigmoi d layer. By alternately arranging multiple convolutional layers and ReLU layers, high-frequency information in shallow features can be extracted, and high-frequency information attention is obtained through the Sigmoi d layer; then, the high-frequency information attention is element-by-element multiplied with the shallow features to obtain high-frequency enhanced features, wherein the schematic diagram of the high-frequency information enhancement unit is as follows Figure 3 shown.

[0086] It should be noted that there may be multiple high-frequency information enhancement modules, the eighth convolutional layer, the ninth convolutional layer, the tenth convolutional layer and the eleventh convolutional layer may be different convolutional layers; the first ReLU layer, the second ReLU layer and the third ReLU layer may be different ReLU layers.

[0087] Again, the high-frequency enhanced features are input into the fourth convolutional layer for feature extraction to obtain deep high-frequency features; and the shallow features are added to the deep high-frequency features, and the addition results are sequentially input into the fifth convolutional layer, the upsampling module and the sixth convolutional layer to restore the size of the original whiteboard image and obtain mixed enhanced features.

[0088] It should be noted that the fourth convolution layer, the fifth convolution layer and the sixth convolution layer may be different convolution layers.

[0089] Feature splicing is achieved by stacking the mixed enhancement features and the high-frequency features in the channel dimension, and feature fusion is achieved through the seventh convolutional layer to obtain fused enhancement features; finally, based on the fused enhancement features and the whiteboard image, a whiteboard enhanced image is output.

[0090] It should be noted that if the simulation process adopts the step of reducing the resolution, then it is necessary to perform up-sampling processing on the whiteboard enhanced image to restore the resolution of the initial whiteboard image.

[0091] Please refer to Figure 4 , Figure 4 It is a structural schematic diagram of an embodiment of a whiteboard image enhancement system provided by the present application, comprising an extraction module 01, an enhancement module 02 and a fusion module 03;

[0092] The extraction module 01 is used to extract features from the acquired whiteboard image to obtain shallow features and high-frequency features respectively;

[0093] Furthermore, the whiteboard image enhancement system also includes: the acquisition module, which is used to acquire the initial whiteboard image and perform simulation processing on the initial whiteboard image to obtain the whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise increase processing, intensity blur processing, resolution reduction processing and other simulation processing methods.

[0094] It should be noted that the initial whiteboard image refers to the original image captured by the image acquisition device. The image contains a whiteboard with background, shadows and writing information. Features rich in spatial information such as background information are shallow features, and features with large grayscale changes such as handwriting are high-frequency features.

[0095] Specifically, the text color processing step is: randomly selecting a scene from the data set of the initial whiteboard image, judging whether the pixel is background or text by calculating the average value of the pixels of the three channels RGB at each pixel position, and changing the pixel value judged as text. When the average value of the pixel value exceeds a certain value, the pixel is judged as background, otherwise it is text, and the pixel value of the text is changed to the pixel value of the selected color to achieve text color change; the overall brightness processing step is: adjusting the pixel value of the image, and then adjusting the overall brightness of the image to simulate scenes in different lighting environments; the noise adding processing step is: adding noise such as Gaussian distribution or Poisson distribution noise to the image; the intensity blur processing step is: using a blur kernel to blur the image, where the blur kernel types include Gaussian blur, isotropic and anisotropic; the resolution reduction processing step is: reducing the image resolution through Bicubic interpolation or Bi li near interpolation.

[0096] In this way, by simulating the initial whiteboard image, the real whiteboard image and its degradation process can be simulated, and a large number of whiteboard image data pairs can be generated.

[0097] After the acquired initial whiteboard image is simulated to obtain a whiteboard image, feature extraction can be performed on the whiteboard image to obtain shallow features and high-frequency features; specifically, the extraction module 01 includes: a first extraction unit and a second extraction unit; the first extraction unit is used to downsample the whiteboard image to obtain a downsampled image, and perform feature extraction on the downsampled image to obtain the shallow features; the second extraction unit is used to perform feature extraction on the whiteboard image to obtain high-frequency features in the whiteboard image.

[0098] It should be noted that the whiteboard image may be downsampled by a downsampling module, wherein the downsampling module may downsample the whiteboard image by 2 or 4 times, and this embodiment does not limit the downsampling multiple.

[0099] It should be noted that shallow features are rich in spatial information, that is, intuitive information such as the position and shape of objects in the image, while high-frequency features refer to features with large grayscale changes such as handwriting in the image.

[0100] It should be noted that feature extraction can be achieved through convolutional layers, and the convolutional layers for extracting shallow features and high-frequency features are different convolutional layers. The shallow features are usually the output of the previous layer or two layers of the convolutional layer network, and the output of the convolutional layer network with three or more layers is the deep feature.

[0101] In this way, by downsampling the whiteboard image, the amount of network calculation can be reduced, and shallow features containing more pixel information and high-frequency features containing high-frequency information such as handwriting can be extracted separately, which facilitates the subsequent different processing methods for shallow features and high-frequency features, and can effectively retain the shallow features and high-frequency information in the whiteboard image.

[0102] The enhancement module 02 is used to perform high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance the high-frequency information in the shallow features;

[0103] The enhancement module includes: a third extraction unit and a processing unit;

[0104] The third extraction unit is used to extract the shallow features to obtain shallow high-frequency features; the processing unit is used to perform high-frequency enhancement processing on the shallow high-frequency features to obtain high-frequency enhanced features;

[0105] It should be noted that the shallow high-frequency features are used as input features, and are alternately set through multiple convolutional layers and ReLU layers in sequence, and the shallow high-frequency information is normalized through the Sigmoid layer to obtain high-frequency information attention. The high-frequency information attention is multiplied element by element with the input feature, i.e., the shallow high-frequency feature, to obtain the high-frequency enhancement feature. Finally, the high-frequency enhancement feature is used as the input feature, and the above operation is repeated multiple times to obtain the final high-frequency enhancement feature. It should be noted that the number of repetitions is 2 times or more.

[0106] It should be noted that each convolution layer and each ReLU layer in the alternating convolution layers and ReLU layers may be different, and this embodiment is not limited to this feature extraction method.

[0107] In this way, by extracting high-frequency information in shallow features, the high-frequency information of shallow features can be effectively enhanced, which can prevent the problem of high-frequency information being lost as the depth of network reasoning increases.

[0108] The fusion module 03 is used to obtain a whiteboard enhanced image based on the high-frequency enhancement feature and the high-frequency feature.

[0109] Specifically, the fusion module includes: a fourth extraction unit, a fifth extraction unit, a fusion unit and an output unit;

[0110] The fourth extraction unit is used to extract the high-frequency enhancement features to obtain deep high-frequency features; wherein the feature extraction of the high-frequency enhancement features can be achieved through a convolution layer;

[0111] The fifth extraction unit is used to add the shallow features and the deep high-frequency features, and perform feature extraction and upsampling processing on the addition result to obtain a mixed enhanced feature; it can be understood that the addition result can be feature extracted through a convolution layer, and the downsampled image can be restored to the size of the original whiteboard image through upsampling processing, and the upsampling multiple corresponds to the downsampling multiple.

[0112] The fusion unit is used to fuse the hybrid enhancement feature with the high-frequency feature to obtain a fused enhancement feature; it can be understood that feature splicing is achieved by stacking the hybrid enhancement feature and the high-frequency feature in the channel dimension, and feature fusion is achieved through the convolution layer. It should be noted that the convolution layer for feature extraction of the enhancement feature, the convolution layer for feature extraction of the addition result, and the convolution layer for feature fusion can be different.

[0113] The output unit is used to reconstruct the fused enhancement features and output a whiteboard enhanced image. It can be understood that by reconstructing the fused enhancement features, the whiteboard enhanced image is directly obtained.

[0114] It should be noted that if the simulation process in the acquisition module adopts a step of reducing the resolution, then it is necessary to perform upsampling or super-resolution reconstruction on the whiteboard enhanced image to restore the resolution of the initial whiteboard image.

[0115] In this way, a whiteboard enhanced image is obtained based on the high-frequency enhancement feature and the high-frequency feature, which reduces the loss of high-frequency information and can effectively remove image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0116] The embodiment of the present application extracts shallow features and high-frequency features respectively, and adopts different processing methods for the shallow features and high-frequency features to effectively retain the shallow features and high-frequency information in the whiteboard image, thereby reducing the loss of high-frequency information caused by the existing whiteboard image enhancement method, and the inability to effectively remove shadows and other problems; by performing high-frequency enhancement processing on the shallow features, the high-frequency information existing in the shallow features can be enhanced, and the problem of high-frequency information being continuously lost as the network reasoning depth deepens can be prevented; finally, a whiteboard enhanced image is obtained based on the high-frequency enhancement features and the high-frequency features, which reduces the loss of high-frequency information while effectively removing image noise and highlights, thereby improving the image enhancement effect of the whiteboard image.

[0117] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A whiteboard image enhancement method, characterized in that: include: Perform feature extraction on the acquired whiteboard image to obtain shallow features and high-frequency features respectively; Performing high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance high-frequency information in the shallow features; Based on the high-frequency enhancement feature and the high-frequency feature, a whiteboard enhanced image is obtained.

2. The whiteboard image enhancement method according to claim 1, characterized in that: Before extracting features from the acquired whiteboard image, it also includes: An initial whiteboard image is acquired, and simulation processing is performed on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, and resolution reduction processing.

3. The whiteboard image enhancement method according to claim 1, characterized in that: The feature extraction is performed on the acquired whiteboard image to obtain shallow features and high-frequency features, respectively, which are specifically: Downsampling the whiteboard image to obtain a downsampled image, and extracting features from the downsampled image to obtain the shallow features; Feature extraction is performed on the whiteboard image to obtain high-frequency features in the whiteboard image.

4. The whiteboard image enhancement method according to claim 1, characterized in that: The high-frequency enhancement processing is performed on the shallow features to obtain high-frequency enhancement features, specifically: Extracting the shallow features to obtain shallow high-frequency features; The shallow high-frequency features are subjected to high-frequency enhancement processing to obtain high-frequency enhanced features.

5. The whiteboard image enhancement method according to claim 1, characterized in that: The step of obtaining a whiteboard enhanced image based on the high-frequency enhancement feature and the high-frequency feature is specifically as follows: Extracting the high-frequency enhancement features to obtain deep high-frequency features; Adding the shallow layer features to the deep layer high frequency features, and performing feature extraction and upsampling processing on the addition result to obtain mixed enhanced features; Fusing the mixed enhancement feature with the high-frequency feature to obtain a fused enhancement feature; The fused enhanced features are reconstructed to output a whiteboard enhanced image.

6. A whiteboard image enhancement system, characterized in that: include: Extraction module, enhancement module and fusion module; The extraction module is used to extract features from the acquired whiteboard image to obtain shallow features and high-frequency features respectively; The enhancement module is used to perform high-frequency enhancement processing on the shallow features to obtain high-frequency enhanced features, wherein the high-frequency enhancement processing can enhance the high-frequency information in the shallow features; The fusion module is used to obtain a whiteboard enhanced image based on the high-frequency enhancement feature and the high-frequency feature.

7. The whiteboard image enhancement system according to claim 6, characterized in that: Also includes: Get module; The acquisition module is used to acquire an initial whiteboard image and perform simulation processing on the initial whiteboard image to obtain a whiteboard image, wherein the simulation processing includes one or more combinations of the following: text color processing, overall brightness processing, noise addition processing, intensity blur processing, and resolution reduction processing.

8. The whiteboard image enhancement system according to claim 6, characterized in that: The extraction module comprises: a first extraction unit and a second extraction unit; The first extraction unit is used to perform downsampling processing on the whiteboard image to obtain a downsampled image, and perform feature extraction on the downsampled image to obtain the shallow feature; The second extraction unit performs feature extraction on the whiteboard image to obtain high-frequency features in the whiteboard image.

9. The whiteboard image enhancement system according to claim 6, characterized in that: The enhancement module includes: a third extraction unit and a processing unit; The third extraction unit is used to extract the shallow features to obtain shallow high-frequency features; The processing unit is used to perform high-frequency enhancement processing on the shallow high-frequency features to obtain high-frequency enhanced features.

10. The whiteboard image enhancement system according to claim 6, characterized in that: The fusion module includes: a fourth extraction unit, a fifth extraction unit, a fusion unit and an output unit; The fourth extraction unit is used to extract the high-frequency enhancement features to obtain deep high-frequency features; The fifth extraction unit is used to add the shallow layer features and the deep layer high-frequency features, and perform feature extraction and upsampling processing on the addition result to obtain a mixed enhanced feature; The fusion unit is used to fuse the mixed enhancement feature with the high-frequency feature to obtain a fused enhancement feature; The output unit is used to reconstruct the fused enhancement features and output a whiteboard enhanced image.

Citation Information

Patent Citations

  • Image sample generation method and device, electronic equipment and storage medium

    CN114693521A

  • Image enhancement method and device and electronic equipment

    CN115471421A

  • Remote sensing image super-resolution reconstruction method based on feature information distillation network

    CN115601236A

  • Image panchromatic sharpening method based on wavelet heuristic and high-frequency enhancement

    CN117151983A

  • Noise reduction and enhancement method for CT (Computed Tomography) positioning plate

    CN117152007A