A control method for interactive pixelated display of arbitrary images on the back screen of a mobile phone

Abstract

This invention discloses a control method for the pixelated display and interaction of arbitrary images on the back screen of a mobile phone, relating to the fields of image processing and display control. The method includes: loading and preprocessing the original image to obtain a preprocessed bitmap; logically dividing the preprocessed bitmap into multiple regions corresponding one-to-one with physical pixels according to the physical resolution of the dot matrix screen; sampling and extracting feature color values ​​from each region; encapsulating all feature color values ​​into a pixel data array; and transmitting the pixel data array to the underlying driver through an asynchronous rendering mechanism to refresh the dot matrix screen display. This invention establishes a precise mapping and intelligent sampling between high-resolution images and low-resolution pixels, preserving the visual features of the original image to the greatest extent on low-resolution screens. Furthermore, it ensures smooth display through mechanisms such as double buffering, achieving the technical effect of converting arbitrary images into personalized pixel maps for display with high quality and smoothness, thus enhancing the personalized interactive experience of mobile devices.

Description

Technical Field

[0001] This invention relates to the field of image processing and display control technology, and in particular to a control method for converting images of any source and any format into pixel images adapted to low-resolution dot matrix screens (such as mobile phone back screens or external pixel screens) and performing efficient and smooth rendering and display, applicable to mobile terminals (such as smartphones). Background Technology

[0002] With the rapid improvement of mobile communication device hardware performance, the performance of core components of smartphones such as processors, imaging systems, and screens has reached a plateau, and hardware homogenization is becoming increasingly apparent. In order to stand out in the fierce market competition, mainstream manufacturers are shifting their R&D focus from a simple "performance race" to in-depth exploration of "personalized interaction" and "emotional experience." Against this backdrop, mobile phone back screens with unique visual styles, or external dot-matrix pixel screens (such as arrays composed of multiple LEDs), have become an emerging medium for personalized expression. These screens usually have lower resolutions (e.g., 16x16 pixels; 32x32 pixels), but their retro pixelated display style is loved by users, who want to transform ordinary photos, artworks, and animated emoticons into personalized pixel patterns and display them on these screens.

[0003] However, existing technical solutions have the following obvious shortcomings in achieving high-quality image pixelation display: 1) Loss of detail due to simple scaling: The most direct method is to scale the high-resolution original image directly to the physical resolution of the target pixel screen. This method is simple to calculate, but it will lead to serious blurring of details and loss of information. For example, a small area in the original image may contain multiple color details. Direct scaling will mix these colors into an average color, causing the generated pixel image to completely lose the features of the original image. The effect is particularly poor when representing human faces and complex textures.

[0004] 2) Lack of targeted optimization, resulting in a harsh display effect: Some existing image processing apps provide pixelated filters that do not take into account the physical display characteristics of low-resolution dot matrix screens (such as non-linear brightness and limited color depth). If their effects are directly applied to display on a real dot matrix screen, problems such as color banding, loss of details in dark areas, and uneven brightness may occur, resulting in a harsh and unnatural visual effect.

[0005] 3) Inefficient processing and display workflow, resulting in poor interactive experience: If image processing, data transmission, screen refresh, and other processes are too tightly coupled or improperly scheduled, stuttering, ghosting, or screen tearing can easily occur when displaying dynamic images (such as GIFs), failing to provide a smooth interactive experience; at the same time, continuous high-load processing will also lead to increased power consumption of mobile devices, affecting battery life.

[0006] Therefore, there is an urgent need in this field for a control method that can intelligently and efficiently convert any image into a high-quality pixel image and achieve smooth and interactive display on low-resolution dot matrix screens such as mobile phone back screens. Summary of the Invention

[0007] The present invention aims to at least solve some of the technical problems existing in the prior art, and to provide a control method for the pixelated display and interaction of arbitrary images on the back screen of a mobile phone.

[0008] This invention provides a control method for interactive pixelated display of arbitrary images on the back screen of a mobile phone, which runs in a mobile terminal system including an application software layer, an operating system kernel layer, and a hardware driver layer. The method includes the following steps: S1. Image preprocessing: In response to user selection instructions, load an original image of any format, decode it into a bitmap, and perform scaling and cropping preprocessing on the bitmap according to the physical resolution of the target dot matrix screen to obtain a preprocessed bitmap. S2. Logical Region Mapping and Segmentation: Based on the physical pixel matrix layout of the target dot matrix screen, the preprocessed bitmap is logically divided into multiple logical regions with the same number and corresponding positions. S3, Regional Feature Color Sampling: Traverse each logical region and extract the feature color value that represents the visual features of the region through a sampling algorithm; S4. Pixel data formatting: Encapsulate all the sampled feature color values ​​in sequence into a pixel data array that conforms to the requirements of the target dot matrix screen driving protocol. S5. Asynchronous rendering and display: The pixel data array is asynchronously transmitted to the underlying driver, which then parses the data and controls the target dot matrix screen to refresh and display at a set frequency.

[0009] Furthermore, the sampling algorithm in step S3 includes center point sampling or neighborhood weighted average sampling; step S5 adopts a double buffering mechanism, including a first buffer for driving the current display and a second buffer for the application layer to fill the next frame data, and the pointers of the two buffers switch during the vertical blanking period.

[0010] Furthermore, after step S1 and before S2, or after step S3 and before S4, a step of gamma correction of the image data is included to adapt to the brightness response characteristics of the target dot matrix screen.

[0011] This invention establishes a precise mapping relationship between high-resolution image regions and low-resolution physical pixels, and employs targeted sampling and optimization strategies to extract the color information that best represents local visual features from the original image, thereby preserving the "spirit" of the original image to the greatest extent possible at extremely low resolution. Combined with an efficient data flow architecture (such as double buffering), it achieves end-to-end low latency from image processing to screen display, ensuring smooth display of dynamic content. This method has the advantages of high fidelity, strong versatility, and high efficiency, greatly enhancing the personalized interactive experience of devices such as mobile phone back screens. Attached Figure Description

[0012] Figure 1 This is a system architecture and data processing flowchart provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of logical region segmentation and center sampling in one embodiment of the present invention; Figure 3 This is a schematic diagram of a double-buffered data exchange mechanism in one embodiment of the present invention; Figure 4 This is a flowchart of the control method provided in an embodiment of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings; the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention.

[0014] The core of this invention lies in constructing a complete and efficient control process that starts from the user selecting any image and ends with the stable and high-fidelity display of the corresponding pixel image on the back screen of the mobile phone (or a similar low-resolution dot matrix screen).

[0015] The following is a detailed explanation using a specific embodiment; assuming the target device is a smartphone equipped with a 16x16 pixel LED dot matrix back screen, the user wants to display a landscape photo (JPG format) with a size of 4000x3000 pixels on the back screen.

[0016] Reference Figure 1 and Figure 4 The control method in this embodiment specifically includes the following steps: Step S101, Image Loading and Adaptive Preprocessing: Users select locally stored landscape photos via a dedicated app on their mobile phone or the system's photo album interface; the application layer (APP) calls the image decoding library provided by the operating system (such as Android's BitmapFactory) to read the image file; after decoding, the system checks the image's Exif information, automatically corrects any possible rotations (such as rotations caused by screen orientation when shooting with a mobile phone), ensures the image is correctly oriented, and generates a standard, correctly oriented bitmap object, denoted as Bitmap_Original.

[0017] Considering that the Bitmap_Original resolution (4000x3000) is much higher than the target pixel screen resolution (16x16), direct subsequent processing would result in huge and unnecessary memory and computational overhead. Therefore, intelligent pre-scaling is required. In this embodiment, the target size for pre-scaling is not directly scaled to 16x16, but rather scaled to an intermediate size that is proportional to the target resolution but larger, such as 256x256 pixels. The advantages of doing so are: firstly, it significantly reduces the amount of bitmap data in memory; secondly, it preserves much richer color and detail information than 16x16 for subsequent accurate region segmentation, which is beneficial to improving sampling quality. The scaling algorithm can use bilinear interpolation to achieve a good balance between efficiency and quality. The pre-processed bitmap is denoted as Bitmap_Scaled.

[0018] Step S102: Establish pixel-level mapping and logical region segmentation: This is one of the key steps of the present invention, which aims to establish a precise mapping between physical pixels and source image areas; each physical LED pixel of the dot matrix screen (a total of 16x16=256) needs to obtain a color value from Bitmap_Scaled; we logically divide Bitmap_Scaled into 256 logical regions that correspond one-to-one with physical pixels.

[0019] The specific calculation is as follows: Let the width of Bitmap_Scaled be W_s (256 pixels) and the height be H_s (256 pixels); the physical width of the target dot matrix screen be W_d (16) and the height be H_d (16); then the width of each logical region block is block_w = W_s / W_d = 256 / 16 = 16 pixels, and the height is block_h = H_s / H_d = 256 / 16 = 16 pixels; thus, Bitmap_Scaled is evenly divided into a grid of 16 rows and 16 columns, and each grid is a logical region R(i, j), where i is the row index (0≤i≤15) and j is the column index (0≤j≤15); the logical region R(i, j) corresponds to the LED pixel in the i-th row and j-th column of the physical screen; Figure 2As shown, the grid on the left represents this logical division.

[0020] Step S103: Feature color sampling based on human visual characteristics: Traversing each logical region R(i, j), we need to extract the "feature color" that best represents the visual characteristics of that region; the simplest method is the "center point sampling method": calculate the geometric center coordinates (Cx, Cy) of region R(i, j), where Cx = j block_w + block_w / 2, Cy = i block_h + block_h / 2; After rounding the coordinates, directly obtain the pixel RGB value of Bitmap_Scaled at the (Cx, Cy) coordinates; This method is fast and works well for areas with uniform color.

[0021] To achieve a smoother and more noise-resistant result, this embodiment preferably uses the "Gaussian weighted average sampling method". This method considers not only the center point but also the pixels in its surrounding neighborhood, but assigns different weights to different pixels. A sampling radius r (e.g., r = 2 pixels) is set with the center point as the origin. For all pixels (x, y) within the region R(i, j) and located in a circular (or square) neighborhood centered at (Cx, Cy) with radius r, the Euclidean distance d to the center is calculated. The Gaussian function G(d) = exp(-d^2 / (2...)... The weight of a pixel is calculated using sigma^2, where sigma is the standard deviation that controls the rate of weight decay. Pixels closer to the center have higher weights, while those farther away have lower weights or are even ignored. Then, the weighted average of the colors of all pixels in the neighborhood is calculated as the feature color Color(i, j) of the region R(i, j). This method can effectively smooth out small noise in the original image and make the color transitions perceived by the human eye more natural. It is especially suitable for images containing smooth transition areas such as gradient skies and skin shadows.

[0022] Step S104, Color Space Conversion and Data Formatting: The sampled feature color Color(i, j) is usually in the standard RGB888 format (8 bits per color channel, 24 bits in total); however, the driver chip of the target dot matrix screen may support different data formats; for example, the common WS2812 series LED driver chip requires the data format to be in GRB order; therefore, the RGB values ​​need to be rearranged according to the driver chip requirements; at the same time, some low-end drivers may only support the RGB565 format (5 bits for red, 6 bits for green, and 5 bits for blue), so the 24-bit color needs to be compressed into 16 bits; this step completes the conversion from general color values ​​to data units of a specific hardware driver protocol.

[0023] After the conversion is complete, all 256 pixel data are encapsulated into a one-dimensional byte array Pixel_Data[] according to the scanning order of the dot matrix screen (usually from left to right, from top to bottom, i.e., row-major order). This array is the complete image data of a frame that will eventually be sent to the hardware for display. If the original image is a GIF animation, steps S101-S104 need to be repeated for each frame to generate a frame data queue and record the display duration (delay time) of each frame.

[0024] Step S105: Achieve asynchronous tear-free rendering through a double-buffering mechanism: This is crucial for ensuring smooth display, especially for dynamic content; such as Figure 3 As shown, the system maintains two buffers of the same size in memory: Buffer_A and Buffer_B; the underlying display driver has a pointer to the current display buffer, Current_Buffer_Ptr; this process is divided into the following three stages: 1) Phase 1 (Frame N Display Period): Current_Buffer_Ptr points to Buffer_A. The driver layer is reading data from Buffer_A and scanning the dot matrix screen at a fixed frequency (e.g., 60Hz) for display. At the same time, the application layer (image processing thread) is processing the next frame image and writing the processed pixel data Pixel_Data_N+1 to Buffer_B.

[0025] 2) Phase 2 (Vertical Blanking Period Switching): After the application layer finishes filling Buffer_B, it sends a synchronization signal to the driver layer. The driver layer does not switch immediately, but waits for the "vertical blanking period" (i.e., the very short interval between two frames, during which the screen does not refresh) after the current frame N scan is completed. During this safe period, the driver layer atomically switches Current_Buffer_Ptr from pointing to Buffer_A to pointing to Buffer_B.

[0026] 3) Phase 3 (Frame N+1 Display Period): Current_Buffer_Ptr now points to Buffer_B, which contains new data; the driver layer begins reading data from Buffer_B and displays frame N+1; at the same time, the application layer can begin writing the data of the next frame to the now-empty Buffer_A.

[0027] This process repeats continuously; the double buffering mechanism ensures the continuity and integrity of the displayed content, completely avoiding screen tearing caused by simultaneous data writing and reading; data can be efficiently transferred from the application layer to the driver layer through shared memory provided by the operating system or character device nodes (such as / dev / pixel_screen).

[0028] Step S106, Low-level driver and hardware execution: The driver layer reads the Pixel_Data[] array from the current buffer (e.g., Buffer_B); according to the specific hardware communication protocol (e.g., SPI, I2C, or single-wire return-to-zero code), the driver layer parses the data of each pixel into a corresponding electrical signal sequence that meets strict timing requirements; for example, for the WS2812 chip, each bit's "0" and "1" needs to be encoded into pulses of different widths; the driver layer generates precise timing through DMA (direct memory access) or a hardware timer, and sends the data stream to the serial data input port of the dot matrix screen through GPIO pins; the control chip on the dot matrix screen receives the data sequentially and latches it into the internal PWM controller to control the brightness and color of each LED, and finally presents the processed pixel image on the physical screen.

[0029] Further optimized embodiments of the present invention include: 1. Dynamic Gamma Correction: Gamma correction can be introduced before sampling in step S103 or after formatting in step S104. Since the brightness of an LED is usually non-linearly related to its driving current, and the human eye's perception of brightness is also non-linear, non-linear mapping of the sampled color values ​​using a lookup table (LUT) (usually using V_out = V_in ^ gamma, where gamma is usually around 2.2) can compensate for these non-linearities, making the final brightness change displayed on the screen more consistent with human eye perception, avoiding the loss of details in dark areas, and resulting in a softer and more natural display effect.

[0030] 2. Adaptation to irregularly shaped screens: The core idea of ​​this method is "mapping-segmentation-sampling". When the dot matrix screen is not arranged in a rectangular pattern, but in a circular, heart-shaped or honeycomb pattern, only the mapping logic in step S102 needs to be modified. The coordinates of the physical pixels (which may be polar coordinates or hexagonal grid coordinates) are mapped back to Bitmap_Scaled, and the corresponding logical region shape (such as a sector or hexagon) is defined. Then, the sampling in step S103 is performed in the non-rectangular region. The core algorithm framework remains unchanged, demonstrating the good scalability of this invention.

[0031] 3. Energy efficiency optimization: For static images, the driver layer can reduce the refresh rate (e.g., to 1Hz); when a change in image content is detected (e.g., switching images, playing GIFs), it can be increased to full speed (e.g., 60Hz); this can significantly reduce the system power consumption when the dot matrix screen is in standby mode.

[0032] Technical effects of specific examples of the present invention: The aforementioned 4000x3000 landscape photo may contain details such as blue sky, mountains, and trees; after processing using the method of this invention, it can be displayed on a 16x16 dot matrix screen. a. The gradient blue area of ​​the sky, after neighborhood weighted averaging and gamma correction, will show a smooth brightness transition, rather than a harsh color block boundary.

[0033] b. The outline of the mountain range, thanks to the precise center sampling that captures the color transition at the boundary between the ridgeline and the sky, can still present a clear outline even at low resolution.

[0034] c. The complex green texture of trees, through the extraction of regional representative colors, can present a layered green area rather than a blurry mass.

[0035] Compared to a blurry mosaic that is directly scaled to 16x16, the pixel image generated by this invention has higher visual recognition and artistic feel, and is closer to the style of "pixel art".

[0036] In summary, this invention cleverly solves the technical challenge of achieving high-quality, high-smoothness visual transformation between high degrees of freedom (arbitrary images) and strong constraints (low-resolution dot matrix screens) through a complete and collaborative software algorithm and control architecture, providing strong technical support for personalized interaction of mobile devices.

[0037] It should be understood that the above description is only a preferred embodiment of the present invention and is not sufficient to limit the technical solution of the present invention. For those skilled in the art, within the spirit and principles of the present invention, additions, subtractions, substitutions, transformations or improvements can be made based on the above description, and all such additions, subtractions, substitutions or improvements should fall within the protection scope of the appended claims of the present invention.

Claims

1. A control method for interactive pixelated display of arbitrary images on the back screen of a mobile phone, running on a mobile terminal including an application layer, a kernel driver layer, and dot matrix screen hardware, characterized in that... The method includes: S1. Image preprocessing steps: Load the original image selected by the user and decode it into a bitmap. Scale and crop the bitmap according to the physical resolution of the target dot matrix screen to generate a preprocessed bitmap. S2. Logical Region Segmentation Step: Based on the number of rows M and columns N of the physical pixel matrix of the target dot matrix screen, the preprocessed bitmap is logically and uniformly divided into M×N logical regions, wherein each logical region is uniquely mapped to a physical pixel on the target dot matrix screen. S3. Feature color sampling step: For each logical region, the feature color value of that region is extracted using a sampling algorithm; S4. Data formatting step: Encapsulate the feature color values ​​of all logical areas into a pixel data array that conforms to the requirements of the target dot matrix screen driving protocol, according to the arrangement order of the physical pixels. S5. Asynchronous rendering and display steps: The pixel data array is transmitted to the kernel driver layer, and the kernel driver layer controls the target dot matrix screen to display at a set refresh rate.

2. The control method for interactive pixelated display of any image on the back screen of a mobile phone according to claim 1, characterized in that, In step S1, scaling the bitmap includes scaling the original bitmap to an intermediate size whose size is an integer multiple of the physical resolution of the target dot matrix screen.

3. The control method for interactive pixelated display of any image on the back screen of a mobile phone according to claim 1, characterized in that, In step S3, the sampling algorithm is a center point sampling method, which includes: calculating the geometric center coordinates of the logical region, and obtaining the color value of the pixel point corresponding to the geometric center coordinates on the preprocessed bitmap as the feature color value.

4. The control method for interactive pixelated display of any image on the back screen of a mobile phone according to claim 1, characterized in that, In step S3, the sampling algorithm is a neighborhood weighted average sampling method, which includes: selecting multiple pixels within a predetermined neighborhood range based on the center point of the logical region; assigning different weights to each pixel based on its distance from the center point, wherein the closer the distance, the higher the weight; calculating the weighted average of the color values ​​of all pixels within the neighborhood range, and using the weighted average as the feature color value.

5. The control method for interactive pixelated display of any image on the back screen of a mobile phone according to claim 1, characterized in that, In step S4, the encapsulation includes converting the feature color value from a first color format to a second color format supported by the target dot matrix screen driver chip.

6. The control method for interactive pixelated display of any image on the back screen of a mobile phone according to claim 1, characterized in that, In step S5, the asynchronous rendering and display step is implemented using a double-buffering mechanism, and the method further includes: Maintain the first and second buffers at the kernel driver layer; While the kernel driver layer reads the current frame data from the first buffer and drives the target dot matrix screen to display, the application layer writes the pixel data array of the next frame into the second buffer. During the vertical blanking period after the current frame ends, the buffer pointer is switched, causing the kernel driver layer to switch to reading the next frame data from the second buffer, while the application layer switches to writing the next frame data to the first buffer.

7. The control method for interactive pixelated display of any image on the back screen of a mobile phone according to claim 1, characterized in that, After step S3, a step of gamma correction is also included, wherein the parameters of the gamma correction are set according to the brightness response characteristics of the target dot matrix screen or the visual characteristics of the human eye.

8. The control method for interactive pixelated display of any image on the back screen of a mobile phone according to claim 1, characterized in that, When the original image is a dynamic image, steps S1 to S4 are executed for each frame of the dynamic image to generate a multi-frame pixel data array and form a frame queue. In step S5, each frame data array is read from the frame queue in sequence for display.

9. The control method for interactive pixelated display of any image on the back screen of a mobile phone according to claim 1, characterized in that, In step S2, when the physical pixels of the target dot matrix screen are arranged in a non-rectangular array, the logical region is divided into geometric shapes corresponding to the non-rectangular array.

10. The control method for interactive pixelated display of any image on the back screen of a mobile phone according to claim 1, characterized in that, The sampling algorithm in step S3 is a color clustering-based sampling method, which includes: performing cluster analysis on the color space of the preprocessed bitmap to obtain K cluster center colors, where K is less than or equal to M×N; and assigning the cluster center colors to the corresponding logical regions according to their spatial position as the feature color values.

Images