Image paper-like processing method and device and electronic equipment

Abstract

The invention relates to the technical field of computers, and discloses an image paper-like processing method and device and electronic equipment. The method comprises the following steps: acquiring two continuous frames of images corresponding to a target interface, wherein the target interface is an interface which contains text content and is about to realize a paper-like effect; performing text screening according to the two continuous frames of images to obtain a text region to be subjected to text enhancement; performing character stroke segmentation processing on the text region to obtain a stroke segmentation result; obtaining a stroke filling value based on the stroke segmentation result; obtaining an initial paper-like picture corresponding to the target interface; and performing enhancement processing on strokes in the initial paper-like picture by adopting the stroke filling value to obtain a final paper-like picture corresponding to the target interface. According to the application, non-native applications, pictures and video contents can achieve a paper-like effect, and the reading experience of an electronic screen is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus and electronic device for paper-like image processing. Background Technology

[0002] Electronic screens have become an indispensable part of people's daily lives and work, but prolonged use can damage eyesight. Achieving paper-like functionality for electronic screens to provide a more comfortable reading experience is a problem that needs to be solved.

[0003] Currently, the problem is addressed primarily from both hardware and software perspectives. Current software-level solutions cannot achieve a paper-like effect for non-native applications, images, and video content. This is because the content and rendering mechanisms of non-native applications are often encapsulated at the application layer, making effective access and processing from the lower levels impossible. Furthermore, images and videos involve a large amount of pixel-level detail and complex visual effects, making unified simulation and processing at the lower level difficult. Therefore, current software-level solutions are primarily designed for paper-like processing of native text content; achieving a paper-like effect for other forms of content remains a challenge. Summary of the Invention

[0004] One objective of this application is to provide a paper-like image processing method, apparatus, and electronic device to solve the technical problem that non-native applications, images, and video content cannot achieve a paper-like effect.

[0005] To address the aforementioned technical problems, one technical solution adopted in this application is to provide a paper-like image processing method, comprising: acquiring two consecutive frames of images corresponding to a target interface, wherein the target interface is an interface containing text content and subject to a paper-like effect; performing text filtering based on the two consecutive frames of images to obtain text regions to be enhanced; performing stroke segmentation processing on the text regions to obtain stroke segmentation results; obtaining stroke fill values ​​based on the stroke segmentation results; acquiring an initial paper-like image corresponding to the target interface; and enhancing the strokes in the initial paper-like image using the stroke fill values ​​to obtain a final paper-like image corresponding to the target interface. By analyzing two consecutive frames of images, the text region can be effectively separated from the entire interface, thus expanding beyond processing native text content to cover text in non-native applications, including text content in images and videos. Furthermore, based on the stroke segmentation results, stroke fill features of each text region are calculated, which can be used to simulate the texture of text on paper media. This application proposes a unified paper-like processing scheme across content formats, which can significantly improve the reading experience on electronic screens.

[0006] In some embodiments, the step of filtering text based on two consecutive frames of images to obtain the text region to be enhanced includes: obtaining the timestamps corresponding to the two consecutive frames of images; calculating the pixel difference between the two consecutive frames of images, marking the regions of pixels with a pixel difference less than a threshold as invariant regions, and marking the regions of pixels with a pixel difference greater than or equal to a threshold as variable regions; calculating the time required for the variable regions to be converted into the invariant regions based on the timestamps; when the time is greater than a preset time threshold, obtaining the area of ​​the invariant region after the variable regions are converted into the invariant regions; when the area of ​​the invariant region is greater than a preset area threshold, then determining the invariant region as the text region to be enhanced. By using dynamic and static region recognition and threshold judgment, selective paper-like processing can be applied to the required text regions, avoiding comprehensive processing of the entire image. This embodiment of the application, through lightweight text detection technology, significantly improves the system's processing performance and resource utilization efficiency while ensuring processing quality.

[0007] In some embodiments, after performing the step of obtaining the text region to be enhanced, the method further includes: detecting the smallest text region within the text region to be enhanced; and obtaining text information within the smallest text region based on the smallest text region. Further detecting the smallest text region allows for subsequent text segmentation based on the smallest text region, thereby improving processing efficiency.

[0008] In some embodiments, obtaining text information in the smallest text region based on the smallest text region includes: preprocessing the image corresponding to the smallest text region, the preprocessing including controlling the length and width of the image corresponding to the smallest text region by a preset scaling ratio; inputting the preprocessed image corresponding to the smallest text region to a preset text detection model; extracting features from the image corresponding to the smallest text region using the text detection model to generate an output feature map; adjusting the height resolution of the output feature map according to the text density of the smallest text region, and determining the width resolution of the output feature map to obtain the adjusted resolution of the output feature map; and extracting the corresponding text information from the output feature map according to the adjusted resolution of the output feature map. Text detection can effectively extract text information from the smallest text region, achieving refined processing of text content; preprocessing can improve the accuracy and stability of subsequent text detection models; processing based on text density can improve the accuracy and effect of text detection; this embodiment can maintain a high detection accuracy while reducing the resource consumption of model computation and improving processing speed.

[0009] In some embodiments, performing stroke segmentation processing on the text region to obtain stroke segmentation results includes: performing stroke segmentation processing on the text information in the smallest text region to obtain stroke segmentation results. Stroke segmentation can provide more accurate character shape analysis and character recognition, further optimizing the effectiveness of text processing and applications.

[0010] In some embodiments, the step of performing stroke segmentation processing on the text information in the smallest text region to obtain stroke segmentation results includes: performing histogram equalization preprocessing on the text region containing the text information in the smallest text region to obtain preprocessed text regions; performing edge detection on the preprocessed text regions to obtain edge detection information corresponding to the text regions; and obtaining stroke-level segmentation results within each text region based on the edge detection information. Here, histogram equalization preprocessing, edge detection, and stroke-level segmentation can improve the quality of the text region image, extract the shape features of the text, and provide more accurate input data for subsequent tasks.

[0011] In some embodiments, before performing the step of enhancing the strokes in the initial paper-like image using the stroke fill value to obtain the final paper-like image, the method further includes: acquiring the latest image of the target interface and the image of the previous moment; comparing the latest image with the previous image to determine whether there is a change region; if there is no change region, then performing the step of enhancing the strokes in the initial paper-like image using the stroke fill value to obtain the final paper-like image corresponding to the target interface. This embodiment optimizes the quality and appearance of the paper-like image, retains the original paper-like effect, and performs corresponding processing based on the presence or absence of change regions, thereby improving the visualization effect and user experience of the final paper-like image.

[0012] In some embodiments, the method further includes: if a change region exists, determining whether the change region overlaps with the stroke region corresponding to the stroke segmentation result; if they do not overlap, performing the enhancement processing on the strokes in the initial paper-like image using the stroke fill value to obtain the final paper-like image; if they overlap, obtaining the stroke region overlapping with the change region, and using the stroke fill value to enhance the uncovered strokes in the initial paper-like image to obtain the final paper-like image corresponding to the target interface. The uncovered strokes refer to strokes in the initial paper-like image that do not overlap with the change region. Specifically, by selectively enhancing the strokes in the paper-like image based on the relationship between the change region and the stroke region, the original appearance of the change region can be preserved while improving the paper-like effect of the non-change region, thereby obtaining the final paper-like image.

[0013] To address the aforementioned technical problems, one technical solution adopted in this application is: providing an image paper-like processing apparatus, comprising: a target image acquisition module, used to acquire two consecutive frames of images corresponding to a target interface, wherein the target interface is an interface containing text content and to be implemented with a paper-like effect; a target text region determination module, used to perform text filtering based on the two consecutive frames of images to obtain a text region to be enhanced; a stroke segmentation module, used to perform stroke segmentation processing on the text region to obtain a stroke segmentation result; a stroke fill value acquisition module, used to obtain a stroke fill value based on the stroke segmentation result; an initial paper-like processing module, used to acquire an initial paper-like image corresponding to the target interface; and a paper-like processing module, used to enhance the strokes in the initial paper-like image using the stroke fill value to obtain a final paper-like image corresponding to the target interface. This image paper-like processing apparatus has the beneficial effects corresponding to the aforementioned image paper-like processing method.

[0014] To address the aforementioned technical problems, one technical solution adopted in this application is to provide an electronic device, including a memory and a processor. The memory is connected to the processor, and the processor is configured to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, it causes the electronic device to implement a paper-like image processing method applied to the electronic device. This electronic device possesses the beneficial effects corresponding to the aforementioned paper-like image processing method applied to the electronic device. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart of a paper-like image processing method provided in an embodiment of this application;

[0017] Figure 2 This is a flowchart of a method for obtaining a text region to be enhanced based on two consecutive frames of images, provided in an embodiment of this application.

[0018] Figure 3 This is a flowchart of a paper-like image processing method provided in another embodiment of this application;

[0019] Figure 4 This is a flowchart of a method for obtaining text information in a minimum text region based on the minimum text region provided in this application embodiment;

[0020] Figure 5 This is a flowchart of a method for performing stroke segmentation processing on text information in the smallest text region to obtain stroke segmentation results, provided in an embodiment of this application.

[0021] Figure 6 This is a flowchart of a paper-like image processing method provided in another embodiment of this application;

[0022] Figure 7 This is the original image corresponding to the target interface provided in the embodiments of this application;

[0023] Figure 8 This is a schematic diagram of the original image corresponding to the target interface after paper-like processing, provided in an embodiment of this application.

[0024] Figure 9This is a schematic diagram of the structure of an image paper-like processing device provided in an embodiment of this application;

[0025] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0027] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, and all are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0028] It should be noted that in the various embodiments described below, there is no necessarily a certain order between the steps. Those skilled in the art can understand from the description of the embodiments of this application that the steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in interchange, etc.

[0029] Paper-like screen technology refers to using technological means to adjust the display effect of electronic screens to more closely resemble the reading experience of traditional paper. Traditional paper has several advantages, such as no glare, low reflectivity, and natural light refraction, which make reading paper documents comfortable for extended periods. However, the characteristics of electronic screens (such as backlighting, reflection, and blue light radiation) can easily cause eye fatigue, visual discomfort, and affect sleep quality. The goal of paper-like screen technology is to simulate the characteristics of paper through software and hardware techniques to improve the reading experience on electronic screens.

[0030] Currently, achieving a paper-like effect on electronic screens is mainly achieved through two levels: hardware and software. At the hardware level, advanced display technologies, such as diffuse reflection and lossless gamma display technology, are needed to reduce glare and improve contrast, presenting a paper-like texture. At the software level, image processing techniques and algorithms can make the interface appear similar to the diffuse reflection caused by the rough surface of paper, giving users the feeling that the screen display is like paper. Hardware provides the basic physical characteristics, and software can further simulate and optimize visuals on this basis, thus bringing users a more comfortable and natural electronic reading experience. Specifically, at the software level, since different operating systems and applications may use different default fonts, obtaining the system's underlying text information ensures that the software uses a consistent font to achieve a unified visual effect. Furthermore, the software needs to know the fonts and font sizes available in the current system to select the appropriate font and size when rendering text content. Therefore, it is necessary to obtain text content information, including font and size, at the system level before rendering the obtained text content information.

[0031] However, the aforementioned software-level solutions present some challenges and limitations in achieving paper-like effects for non-native applications, images, and video content. For non-native applications, their content and rendering mechanisms are typically restricted by the application layer. This means it's impossible to directly obtain text information and perform unified paper-like processing from the system's underlying layers. The content and rendering methods of non-native applications are controlled by the application itself and may not be affected by system-level paper-like features. Therefore, achieving paper-like effects in non-native applications may be somewhat limited. For image and video content, they involve a large amount of pixel-level detail and complex visual effects. Paper-like effects are primarily for text content, but for images and videos, simulating the texture and feel of paper is not easy and may lead to visual distortion or quality loss. Therefore, performing unified simulation and processing at the underlying level is challenging for image and video content.

[0032] To address the aforementioned problems, embodiments of this application provide a method, apparatus, and electronic device for paper-like image processing. The solution provided by these embodiments enables non-native applications, images, and video content to achieve a paper-like effect. The process primarily involves acquiring two consecutive frames of an image corresponding to a target interface. This target interface can be a non-native application, an image, or a video content. These two frames can be obtained by capturing application windows, screen recording, or similar methods. Text filtering and text region recognition are then performed on the acquired frames to determine the text regions to be enhanced. Next, the text regions are segmented into strokes. Based on the stroke segmentation results, a fill value is obtained for each stroke. This fill value can be a value representing the color of the stroke region in a paper-like image, used to simulate the color or texture of the text strokes in a paper-like effect. It can also be a texture image or a texture generated by a texture generation algorithm. Next, an initial paper-like image is obtained corresponding to the target interface. This can be the original image without paper-like processing or a basic paper-like effect. The previously acquired stroke fill values ​​are then applied to enhance each stroke in the initial paper-like image. After stroke enhancement, the final paper-like image corresponding to the target interface is obtained, which includes the paper-like effect and is combined with the original content. The solution provided in this application, by analyzing two consecutive frames of images, can effectively separate text regions from the entire interface, thereby enabling the processing of text content in non-native applications, including text, images, and videos. This method expands the application scope of the paper-like effect, extending it beyond native text content. Simultaneously, by calculating the stroke filling features of each text region based on the stroke segmentation results, the texture of text on printed media can be simulated. These features can include color, texture, lighting effects, etc., to enhance the visual effect of text in the paper-like image and improve the reading experience. Therefore, this application, through this unified paper-like processing solution across content formats, can simulate the reading feel of printed media on an electronic screen, providing users with a more comfortable and natural reading experience. The application of this method can cover various types of content, including text, images, and video content in native applications, creating a more immersive and realistic reading environment for users.

[0033] The paper-like image processing method and apparatus provided in this application can be applied to various devices and are suitable for multiple scenarios. For example, smartphones and tablets: through applications or browsers, a paper-like effect can be achieved on mobile devices, enhancing the reading experience. E-readers: applying the paper-like effect to e-readers makes e-books feel closer to the reading experience of paper books. Smart TVs and monitors: by applying the paper-like effect to smart TVs and large-screen displays, a more comfortable and realistic viewing experience is provided.

[0034] Optionally, the paper-like effect provided in this application embodiment can be activated or deactivated via a function key or switch to provide user control. For example, a switch or function key is provided in the user interface of the application or device to activate or deactivate the paper-like mode; when the user turns on the switch or presses the function key, the application or device switches to the paper-like mode and applies the paper-like effect; when the user turns off the switch or presses the function key again, the paper-like mode is disabled, and the effect reverts to normal mode. This allows users to freely choose whether to enable the paper-like effect according to specific scenarios and personal preferences, thus balancing the user's needs for reading experience and information display, and providing more flexible and personalized functionality.

[0035] The paper-like image processing method proposed in this application is described below through specific embodiments.

[0036] Please see Figure 1 , Figure 1 This is a flowchart of a paper-like image processing method provided in an embodiment of this application. The method includes:

[0037] Step S101: Obtain two consecutive frames of images corresponding to the target interface, wherein the target interface is an interface containing text content and a paper-like effect to be implemented.

[0038] The target interface is a screen that needs to have a paper-like effect applied, containing text content. Specifically, the target interface refers to the final output image with a paper-like effect; this interface may originate from an application, webpage, or other digital interface. The target interface contains text content, which is the area on the interface that requires text enhancement.

[0039] Two consecutive image frames refer to two screenshots or video frames of the target interface taken at two consecutive points in time. These two images need to be captured consecutively, meaning they are adjacent in time with a short time interval, thus allowing the acquisition of information about changes in the target interface within a short period. Both images originate from the same target interface, differing only in the time points at which they were captured. Both images contain the entire content of the target interface, including text areas that require paper-like rendering. In this embodiment, the purpose of acquiring two image frames is for subsequent text filtering and analysis, enabling better location of text areas by utilizing the dynamic changes in interface elements.

[0040] Methods for obtaining two consecutive frames of a target interface include screen capture, which can be done using the operating system's built-in screenshot tool or third-party screenshot software, taking two consecutive screenshots with a sufficiently short time interval between them, such as within one second; and video recording, which can be done using screen recording software or a webcam to record a video stream of the target interface, and then extracting two consecutive video frames from the recorded video as the two frames. Alternatively, emulator recording can be used to obtain the two frames.

[0041] Step S102: Perform text filtering based on the two consecutive frames of images to obtain the text region to be enhanced.

[0042] The text region to be enhanced is a region that exists in two consecutive frames of images. In this embodiment, the dynamic change information of two consecutive frames of the target interface is used to accurately locate stable text regions. First, the timestamps of the two frames are obtained, and the difference value of each pixel in the two frames is calculated. Regions with differences less than a threshold are marked as "unchanging regions," i.e., static regions; regions with differences greater than or equal to the threshold are marked as "changing regions," i.e., dynamic regions. Then, based on the timestamps, the time required for a changing region to become an unchanging region is calculated. Unchanging regions (i.e., static regions) with a duration exceeding a preset threshold and an area greater than a preset value are identified as the target regions for text enhancement. This method, through dynamic and static region identification and threshold judgment, can selectively perform paper-like processing on the required text regions, avoiding comprehensive processing of the entire screen. Through lightweight text detection technology, the system's processing performance and resource utilization efficiency are significantly improved while ensuring processing quality. Furthermore, by accurately identifying relatively stable text content on the interface and avoiding processing dynamically changing non-text elements, valuable text region information is provided for subsequent paper-like processing. For details, please refer to [link to relevant documentation]. Figure 2 Based on the two consecutive image frames, text filtering is performed to obtain the text region to be enhanced, including:

[0043] Step S1021: Obtain the timestamps corresponding to the two consecutive image frames. The timestamp refers to the date and time information when each image frame was captured.

[0044] Step S1022: Calculate the pixel difference between the two consecutive frames of images, mark the regions of pixels with a pixel difference less than a threshold as unchanged regions in the two consecutive frames of images, and mark the regions of pixels with a pixel difference greater than or equal to the threshold as changed regions in the two consecutive frames of images.

[0045] The process involves comparing two frames of images pixel by pixel, calculating the difference between each pixel in the two frames, and marking the regions of pixels with a difference less than a preset threshold as "unchanging regions" (i.e., static regions), and marking the regions with a difference greater than or equal to the threshold as "changing regions" (i.e., dynamic regions).

[0046] The threshold can be adjusted and optimized based on factors such as the actual application scenario and image quality. If the threshold is set too low, even minute noise or slight brightness changes may be identified as "changing regions," potentially leading to the incorrect exclusion of stable text regions. Conversely, if the threshold is set too high, even obvious dynamic changes may be misjudged as "unchanging regions," resulting in the incorrect retention of parts that should not be identified as text regions. Therefore, it is necessary to determine a suitable threshold that can effectively distinguish between static and dynamically changing regions. This threshold can be determined by combining background knowledge, image features, and other factors, and can be achieved through practical testing and repeated adjustments. Alternatively, an adaptive thresholding method can be used, dynamically adjusting the threshold according to the characteristics of different image regions to improve overall recognition accuracy.

[0047] Step S1023: Based on the timestamp, calculate the time required for the changed region to be converted into the unchanged region.

[0048] For example, for each pixel region identified as a "changing region," the timestamp of its first identification is recorded. This "changing region" is then continuously monitored until it is determined to be a "non-changing region" (static text region), and the timestamp of this transition point is recorded. The time difference between these two timestamps yields the time required for the "changing region" to transition from dynamic to static. By performing similar time statistics on all "changing regions," metrics such as the average time, longest time, and shortest time for dynamic text regions to transition to static text regions across the entire image can be obtained.

[0049] Step S1024: When the time is greater than a preset time threshold, obtain the area of ​​the unchanging region after the changed region is converted into the unchanging region.

[0050] This allows for the acquisition of the coordinate range and number of pixels of the "unchanging region," and the calculation of its area. When there are multiple unchanging regions exceeding a preset time threshold, the areas of all these "unchanging regions" can be statistically analyzed. Furthermore, the maximum and minimum areas among these unchanging regions can be calculated.

[0051] Step S1025: When the area of ​​the unchanging region is greater than a preset area threshold, the unchanging region is determined as the text region to be enhanced.

[0052] If there are multiple unchanging regions whose area is greater than a preset area threshold, then these multiple unchanging regions can be identified as the text regions to be enhanced.

[0053] In practical applications, when text is static in an image, users often have higher requirements for text clarity. Therefore, it is necessary to focus on these static text areas that meet the area threshold for subsequent text enhancement processing. This embodiment can effectively target the characteristics of static text areas, adopting a more refined enhancement strategy to meet users' needs for text clarity and improve the overall user experience.

[0054] Step S103: Perform stroke segmentation processing on the text region to obtain stroke segmentation results.

[0055] This process involves breaking down each character in the text region into basic stroke components. Stroke segmentation allows for the acquisition of finer-grained text information. The results of stroke segmentation can include: stroke information for each character, such as the character outline being divided into several basic stroke components, and the geometric features of each stroke, including its starting point, ending point, and direction; information about the relationships between strokes, such as the order of different strokes, the connection relationships between strokes, and the combination of strokes within the same character; and additional stroke feature parameters, such as the thickness and length of each stroke.

[0056] One approach is to first use Optical Character Recognition (OCR) technology to identify characters in the text region, and then further subdivide and segment the characters based on their outlines or stroke information. Alternatively, a dedicated deep learning model can be trained to take a text image as input and output the stroke segmentation results for each character.

[0057] Step S104: Based on the stroke segmentation results, obtain the stroke fill value.

[0058] The obtained stroke information can be analyzed and quantified to calculate the thickness, length, direction and other features of each stroke, and converted into numerical stroke fill values. These stroke fill values ​​can be used as input parameters for subsequent enhancement processing.

[0059] In this embodiment, stroke fill values ​​can be obtained using the mean value scheme and the contrast color scheme. The mean value scheme involves: based on the previously obtained stroke segmentation results, performing an erosion operation on the stroke region to obtain the center position of the stroke; then, taking the average value of the pixels at these center positions on the original input image as the stroke fill value; this scheme can maintain the original stroke color unchanged, only uniformly enhancing it. The contrast color scheme involves: also based on the stroke segmentation results, obtaining the RGB average value of the background region; converting the RGB values ​​to the YUV color space; calculating the complementary color value based on the YUV values; converting the complementary color value back to RGB as the new stroke fill color; this scheme can explicitly modify the stroke color, making it significantly contrast with the background. RGB is an additive color mixing model that uses three basic color channels (red, green, and blue) to represent and describe colors, with each channel typically ranging from 0 to 255. The YUV color space is a color encoding scheme mainly used in video and television broadcasting; Y represents the luminance channel, and U and V represent the chrominance channels. The Y channel represents grayscale information, while the U and V channels represent color information.

[0060] In this embodiment, the stroke fill value is either the average pixel value at the center position of the stroke in the original input image, or a new RGB value that is calculated to form a complementary color with the background color.

[0061] Step S105: Obtain the initial paper-like screen corresponding to the target interface.

[0062] The initial paper-like image can be generated using certain image processing algorithms or obtained from other sources. Alternatively, the initial paper-like image can be obtained by directly merging the stroke segmentation layer and the background layer.

[0063] For example, obtain the stroke segmentation result to get the binarized or segmentation mask of the stroke region; obtain the original background image; directly overlay and blend the stroke segmentation layer and the background layer; in the stroke region, use the color value of the original stroke, and in the non-stroke region, use the original color value of the background image, thereby obtaining the initial paper-like image.

[0064] For example, the original image is converted to RGBA (Red, Green, Blue, Alpha, with red, green, and blue alpha channels) format. This original image can be one frame from two consecutive images. The red (R), green (G), and blue (B) color channels are extracted from the original RGB image, and an alpha (A) channel is added to represent the image's transparency. These four RGBA channels are then merged to form a single RGBA image. Next, the paper layer is also converted to RGBA format, meaning the R, G, B, and A channels are extracted from the paper image and combined into an RGBA image. Finally, the original image RGBA layer and the paper RGBA layer are blended at a certain ratio to obtain the initial paper-like image.

[0065] Step S106: Use the stroke fill value to enhance the strokes in the initial paper-like image to obtain the final paper-like image corresponding to the target interface.

[0066] In this process, the initial paper-like image is enhanced using the previously obtained stroke fill values.

[0067] As we know, most human eyes sample at a frequency of 24 frames per second. When the frame rate exceeds 24 frames per second, the human eye perceives the image as dynamic; when the frame rate is below 24 frames per second, the human eye perceives the image as static or choppy. Furthermore, if the stroke area is directly set to the target value, the stroke area between two consecutive frames will change significantly, which can be jarring to the user. Therefore, this embodiment proposes an optimization strategy: combining the screen refresh rate N (N>24 frames per second), a gradient stroke filling strategy is adopted; the stroke internal mean scheme and the contrast color scheme use similar transition strategies; taking the stroke internal mean scheme as an example, the gradient stroke fill value is set in the RGB space according to the screen refresh rate N. This achieves a smooth display effect before and after text enhancement, reducing the jarring effect perceived by the user.

[0068] Specifically, firstly, the stroke fill value is obtained:

[0069] Mean averaging method: Perform erosion operation on the stroke segmentation image to obtain the mean RGB value of the pixels at the center of the stroke in the original image. mean [1*3] .

[0070] Contrast color scheme: Obtain the average RGB value of the background area, convert it to YUV space, and obtain its contrast color YUV based on the Y value. complent [1*3] .

[0071] Then, a gradient fill strategy based on the screen refresh rate N (N>24):

[0072] The mean-averaging scheme corresponds to: in the RGB space, based on the index step of the current frame, using RGB... mean [1*3] Using RGBXY as the target value and linear interpolation as the starting value, the fill value RGB' for each frame is calculated:

[0073] RGB' = RGB XY +step*(RGB mean [1*3] -RGB XY ) / N;

[0074] RGB XY RGB' represents the original values ​​of the stroke area, RGB' represents the values ​​to be filled in the stroke area, step represents the index of the current frame, and N represents the screen refresh rate.

[0075] The contrast color scheme corresponds to: in the YUV space, based on the index step of the current frame, using YUV... complent [1*3] The Y value is the target value. Starting with the Y value of the original pixel, the Y' value of each frame is calculated using linear interpolation.

[0076] Y'=(Y XY +step*(YUV complent [1*3] -Y XY ) / N) / 180;

[0077] Among them, Y XY It is the Y value of YUV, similar to the original Y value of the stroke area in RGB. The original Y value of the stroke area can refer to the brightness value of the pixel corresponding to the text or edge in the RGB image.

[0078] Finally, the calculated gradient fill value RGB' is applied to the stroke area of ​​the initial paper-like image to achieve a smooth stroke enhancement effect.

[0079] In this embodiment, by employing a gradient fill strategy based on the screen refresh rate, a smooth transition in the stroke area can be achieved, reducing the abruptness perceived by the user. Whether using an average or contrasting color scheme, a smooth stroke enhancement effect can be achieved.

[0080] The paper-like image processing method provided in this application, by analyzing two consecutive image frames, can effectively separate text regions from the entire interface. This expands its scope beyond native text content to include text in non-native applications, such as images and videos. Furthermore, based on stroke segmentation results, the stroke filling features of each text region are calculated, which can be used to simulate the texture of text on printed media. This application proposes a unified paper-like processing scheme across content formats, significantly improving the reading experience on electronic screens.

[0081] Please see Figure 3 , Figure 3 This is a flowchart of a paper-like image processing method provided in another embodiment of this application. Figure 3 The corresponding methods and Figure 1 The difference between the corresponding methods is that, Figure 3 The corresponding implementation adds steps of text filtering and minimum text region detection to accurately locate the regions requiring text enhancement. The method includes:

[0082] Step S201: Obtain two consecutive frames of images corresponding to the target interface, wherein the target interface is an interface containing text content and a paper-like effect to be implemented.

[0083] Step S202: Based on the two consecutive frames of images, perform text filtering to obtain the text region to be enhanced.

[0084] For details of steps S201 and S202, please refer to the above method embodiments.

[0085] Step S203: Detect the smallest text region in the text region to be enhanced.

[0086] According to the above method embodiment, the area of ​​the text region can be obtained, and the text region with the smallest area is the minimum text region. For this minimum text region, its corresponding bounding rectangle can be generated. Based on this bounding rectangle, the corresponding region is extracted from the original image. The original image can be the frame image most recent in time among two consecutive frames.

[0087] By focusing on the densest area of ​​text content, i.e. the smallest text area, subsequent text enhancement processing can be performed. This allows for more precise control over the text enhancement effect, improves the overall display quality, and enhances processing efficiency.

[0088] Step S204: Obtain the text information in the minimum text region based on the minimum text region.

[0089] Please see Figure 4Obtaining the text information within the smallest text region, based on the smallest text region, includes:

[0090] Step S2041: Preprocess the image corresponding to the smallest text region. The preprocessing includes controlling the length and width of the image corresponding to the smallest text region by a preset scaling ratio.

[0091] Step S2042: Input the image corresponding to the preprocessed smallest text region into the preset text detection model;

[0092] Step S2043: Extract features from the image corresponding to the smallest text region using the text detection model to generate an output feature map;

[0093] Step S2044: Adjust the height resolution of the output feature map according to the text density of the minimum text region, and determine the width resolution of the output feature map to obtain the adjusted resolution of the output feature map;

[0094] Step S2045: Extract the corresponding text information from the output feature map according to the resolution of the adjusted output feature map.

[0095] For example, suppose the input image resolution is 1920x1080 pixels. The original solution is: input resolution 1920x1080, output resolution 480x270 (1 / 4 scaling), output aspect ratio 16:9. The optimized solution is: input resolution remains 1920x1080, output resolution 480x135 (1 / 4 width, 1 / 8 height), output aspect ratio 32:9 (twice the original). In scenarios such as UI interfaces or picture books, the sensitivity to text height is relatively low. Therefore, while maintaining text detection accuracy, the output height can be further reduced to half of the original, i.e., 135 pixels.

[0096] This reduces the model's computational resource consumption and improves processing speed; the output aspect ratio is increased to twice the original, making it more suitable for scenarios with lower text density; and because text height is less sensitive, this reduction in height will not significantly affect the accuracy of text detection.

[0097] In this embodiment, the text detection process has been optimized. Text detection can effectively extract text information from the smallest text regions, enabling refined processing of text content. Preprocessing improves the accuracy and stability of the subsequent text detection model; processing based on text density further enhances the accuracy and effectiveness of text detection. This optimization scheme maintains high detection accuracy while reducing model resource consumption and increasing processing speed.

[0098] Step S205: Perform stroke segmentation processing on the text information in the smallest text region to obtain stroke segmentation results. Stroke segmentation can provide more accurate character shape analysis and character recognition, further optimizing the effect of text processing and application.

[0099] Please see Figure 5 The text information in the smallest text region is processed by stroke segmentation to obtain stroke segmentation results, including:

[0100] Step S2051: Perform histogram equalization preprocessing on the text region containing the text information in the smallest text region to obtain the preprocessed text region;

[0101] Step S2052: Perform edge detection on the preprocessed text region to obtain the edge detection information corresponding to the text region;

[0102] Step S2053: Based on the edge detection information, obtain the stroke-level segmentation results within each text region.

[0103] Histogram equalization effectively stretches the image's grayscale histogram, highlighting details in text regions. Edge detection on the preprocessed text regions allows for the acquisition of the outline information of strokes within those regions.

[0104] The edge detection information obtained earlier can be used to further analyze and extract stroke-level information within each text region. For example, by performing clustering and connected component analysis on the edge detection results, interconnected edge segments can be identified as the same stroke, ultimately outputting the stroke-level segmentation results within each text region.

[0105] This embodiment improves the quality of text region images, extracts the shape features of text, and provides more accurate input data for subsequent tasks through histogram equalization preprocessing, edge detection, and stroke-level segmentation.

[0106] Step S206: Based on the stroke segmentation results, obtain the stroke fill value.

[0107] Step S207: Obtain the initial paper-like screen corresponding to the target interface.

[0108] Step S208: Use the stroke fill value to enhance the strokes in the initial paper-like image to obtain the final paper-like image corresponding to the target interface.

[0109] For details of steps S206 to S208, please refer to the above method embodiments.

[0110] Please see Figure 6 , Figure 6 This is a flowchart of a paper-like image processing method according to another embodiment of this application. The method includes:

[0111] Step S301: Obtain two consecutive frames of images corresponding to the target interface, wherein the target interface is an interface containing text content and a paper-like effect to be implemented.

[0112] Step S302: Perform text filtering based on the two consecutive frames of images to obtain the text region to be enhanced.

[0113] Step S303: Perform stroke segmentation processing on the text region to obtain stroke segmentation results.

[0114] Step S304: Based on the stroke segmentation results, obtain the stroke fill value.

[0115] Step S305: Obtain the initial paper-like screen corresponding to the target interface.

[0116] Step S306: Obtain the latest image of the target interface and the image of the previous moment.

[0117] Step S307: Compare the image at the latest moment with the image at the previous moment to determine whether there is a changed region.

[0118] If no changing region exists, proceed to step S308. If a changing region exists, proceed to step S309.

[0119] Step S308: Use the stroke fill value to enhance the strokes in the initial paper-like image to obtain the final paper-like image corresponding to the target interface.

[0120] Step S309: Determine whether the changed area overlaps with the stroke area corresponding to the stroke segmentation result;

[0121] If they do not overlap, proceed to step S308. If they overlap, proceed to steps S310 and S311.

[0122] Step S310: Obtain the stroke region that overlaps with the changed region.

[0123] Step S311: Using the stroke fill value, enhance the strokes that are not covered in the initial paper-like image to obtain the final paper-like image corresponding to the target interface.

[0124] The uncovered strokes refer to strokes in the initial paper-like image that do not overlap with the changed area.

[0125] The main difference between this embodiment and the above-described method embodiment lies in that steps S306 and S307 compare the image at the latest moment with the image at the previous moment to determine whether there is a changed region. If no changed region exists, the initial paper-like image is directly enhanced with strokes. If a changed region exists, it is necessary to further determine whether the changed region overlaps with the stroke segmentation region. If the changed region overlaps with the stroke segmentation region, the stroke region overlapping with the changed region is not enhanced; instead, only the other unchanged stroke regions are enhanced. This avoids repeatedly enhancing already changed regions, preventing inconsistent effects. Furthermore, based on the relationship between the changed region and the stroke region, strokes in the paper-like image are selectively enhanced, thus preserving the original appearance of the changed region while improving the paper-like effect of the non-changed regions, thereby obtaining the final paper-like image.

[0126] It should be noted that the steps in the different flowcharts of the above embodiments can be combined according to the needs of the scenario.

[0127] Please see Figure 7 and Figure 8 , Figure 7 It is the original image corresponding to the target interface. Figure 8 This is a schematic diagram of the original image corresponding to the target interface after paper-like processing. The text in the original image was enhanced using the aforementioned stroke-level averaging scheme. By comparing these two schematic diagrams, it is clear that the method in this embodiment employs precise stroke segmentation and stroke filling techniques, ensuring clear text content and improving the paper-like effect while maintaining the background content.

[0128] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of an image paper-like processing apparatus provided in an embodiment of this application. The apparatus 40 includes:

[0129] The target image acquisition module 41 is used to acquire two consecutive frames of images corresponding to the target interface, wherein the target interface is an interface containing text content and a paper-like effect to be implemented.

[0130] The target text region determination module 42 is used to perform text filtering based on the two consecutive frames of images to obtain the text region to be enhanced.

[0131] The stroke segmentation module 43 is used to perform stroke segmentation processing on the text region to obtain stroke segmentation results;

[0132] The stroke fill value acquisition module 44 is used to obtain the stroke fill value based on the stroke segmentation result;

[0133] The initial paper-like module 45 is used to acquire the initial paper-like screen corresponding to the target interface;

[0134] The paper-like processing module 46 is used to enhance the strokes in the initial paper-like image using the stroke fill value to obtain the final paper-like image corresponding to the target interface.

[0135] The image paper-like processing device 40 can be a software module. The software module includes several instructions, which are stored in a memory. The processor can access the memory and call the instructions to execute them, so as to complete the image paper-like processing method described in the above embodiments.

[0136] In some embodiments, the image paper-like processing apparatus 40 can also be constructed from hardware devices. For example, the image paper-like processing apparatus 40 can be constructed from one or more chips, and the chips can work in coordination to complete the image paper-like processing method described in the various embodiments above. As another example, the image paper-like processing apparatus 40 can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0137] It should be noted that the image paper-like processing apparatus 40 described above can execute the image paper-like processing method for electronic devices provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the embodiments of the image paper-like processing apparatus 40 can be found in the image paper-like processing method for electronic devices provided in the embodiments of this application.

[0138] See Figure 10 , Figure 10 This is a schematic diagram of the structure of an electronic device 50 provided in an embodiment of this application. The electronic device 50 includes one or more processors 51 and a memory 52. ​​The memory 52 is connected to one or more processors 51, for example, via a bus.

[0139] Processor 51 is configured to support the electronic device 50 in performing the corresponding functions in the methods described in the above method embodiments. Processor 51 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0140] Memory 52 is used to store program code, etc. Memory 52 may include volatile memory (VM), such as random access memory (RAM); memory may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 52 may also include combinations of the above types of memory.

[0141] The memory 52 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the image paper-like processing method in the embodiments of this application. The processor 51 executes various functional applications and data processing of the image paper-like processing method and the image paper-like processing device by running the non-volatile software programs, instructions, and modules stored in the memory, that is, it realizes the functions of each module or unit of the image paper-like processing method and the image paper-like processing device provided in the above method embodiments.

[0142] The memory 52 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created based on the use of the image paper-like processing apparatus. In some embodiments, the memory 52 may optionally include memory remotely located relative to the processor, which can be connected to the image paper-like processing apparatus via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0143] The one or more modules are stored in the memory 52. ​​When executed by the one or more processors 51, they perform the paper-like image processing method in any of the above method embodiments. For example, they perform the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.

[0144] The electronic device 50 in this application embodiment may specifically be a computer, tablet computer, e-reader, smartphone, etc.

[0145] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method described in the foregoing embodiments.

[0146] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0147] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for paper-like image processing, characterized in that, include: Acquire two consecutive frames of images corresponding to the target interface, wherein the target interface is an interface containing text content and is intended to achieve a paper-like effect; Text filtering is performed based on the two consecutive images to obtain the text region to be enhanced. The text region is processed by stroke segmentation of the characters to obtain stroke segmentation results; Based on the stroke segmentation results, the stroke fill value is obtained; Obtain the initial paper-like screen corresponding to the target interface; The strokes in the initial paper-like image are enhanced using the stroke fill value to obtain the final paper-like image corresponding to the target interface.

2. The method according to claim 1, characterized in that, The step of filtering text based on the two consecutive image frames to obtain the text region to be enhanced includes: Obtain the timestamps corresponding to the two consecutive frames of images; Calculate the pixel difference between two consecutive frames of images, mark the regions of pixels with a pixel difference less than a threshold as unchanged regions in the two consecutive frames of images, and mark the regions of pixels with a pixel difference greater than or equal to the threshold as changed regions in the two consecutive frames of images. Based on the timestamp, calculate the time required for the changed region to transform into the unchanged region; When the time exceeds a preset time threshold, the area of ​​the unchanged region after the changed region is converted into the unchanged region is obtained; When the area of ​​the unchanging region is greater than a preset area threshold, the unchanging region is determined as the text region to be enhanced.

3. The method according to claim 1, characterized in that, After performing the step of obtaining the text region to be enhanced, the method further includes: Detect the smallest text region in the text region to be enhanced; Based on the minimum text region, obtain the text information in the minimum text region.

4. The method according to claim 3, characterized in that, The step of obtaining the text information in the minimum text region based on the minimum text region includes: The image corresponding to the smallest text region is preprocessed, and the preprocessing includes controlling the length and width of the image corresponding to the smallest text region by a preset scaling ratio. Input the image corresponding to the preprocessed smallest text region into the preset text detection model; The text detection model is used to extract features from the image corresponding to the smallest text region to generate an output feature map. Based on the text density of the minimum text region, the height resolution of the output feature map is adjusted, and the width resolution of the output feature map is determined to obtain the resolution of the adjusted output feature map. Based on the resolution of the adjusted output feature map, extract the corresponding text information from the output feature map.

5. The method according to claim 3 or 4, characterized in that, The step of performing stroke segmentation processing on the text region to obtain stroke segmentation results includes: The text information in the smallest text region is processed by stroke segmentation to obtain the stroke segmentation result.

6. The method according to claim 5, characterized in that, The step of performing stroke segmentation processing on the text information in the smallest text region to obtain stroke segmentation results includes: Histogram equalization preprocessing is performed on the text regions containing text information in the smallest text region to obtain the preprocessed text regions. Edge detection is performed on the preprocessed text region to obtain the edge detection information corresponding to the text region; Based on the edge detection information, the stroke-level segmentation results within each text region are obtained.

7. The method according to claim 1, characterized in that, Before performing the step of enhancing the strokes in the initial paper-like image using the stroke fill value to obtain the final paper-like image, the method further includes: Obtain the latest image of the target interface, and the image of the previous moment of the latest moment; Compare the image at the latest moment with the image at the previous moment to determine if there is a region of change; If there is no area of ​​change, then the strokes in the initial paper-like image are enhanced using the stroke fill value to obtain the final paper-like image corresponding to the target interface.

8. The method according to claim 7, characterized in that, The method further includes: If there is a variable region, determine whether the variable region overlaps with the stroke region corresponding to the stroke segmentation result; If there is no overlap, the strokes in the initial paper-like image are enhanced using the stroke fill value to obtain the final paper-like image corresponding to the target interface. If there is an overlap, the stroke area that overlaps with the change area is obtained, and the stroke fill value is used to enhance the strokes that are not covered in the initial paper-like image to obtain the final paper-like image corresponding to the target interface. The strokes that are not covered refer to the strokes that do not overlap with the change area in the initial paper-like image.

9. An image paper-like processing apparatus, characterized in that, include: The target image acquisition module is used to acquire two consecutive frames of images corresponding to the target interface, wherein the target interface is an interface containing text content that is to be implemented with a paper-like effect. The target text region determination module is used to perform text filtering based on the two consecutive frames of images to obtain the text region to be enhanced. The stroke segmentation module is used to perform stroke segmentation processing on the text region to obtain stroke segmentation results; The stroke fill value acquisition module is used to obtain the stroke fill value based on the stroke segmentation result; An initial paper-like module is used to acquire the initial paper-like screen corresponding to the target interface; The paper-like processing module is used to enhance the strokes in the initial paper-like image using the stroke fill value to obtain the final paper-like image corresponding to the target interface.

10. An electronic device, characterized in that, include: A memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor, when executing the one or more computer programs, causing the electronic device to perform the method as described in any one of claims 1-8.

Images