Tablet computer control method and system based on image super-resolution technology
By using image super-resolution technology to perform super-resolution processing and coordinate mapping on the touch screen operation of tablet computers, the problem of insufficient touch screen resolution is solved, and the operation accuracy and user experience are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN INCAR TECH CO LTD
- Filing Date
- 2025-01-20
- Publication Date
- 2026-06-26
AI Technical Summary
Existing touchscreen technology cannot provide sufficient resolution for applications such as professional drawing and precision control, causing difficulties for users when performing delicate operations.
A tablet computer control method based on image super-resolution technology is adopted. The original image is super-resolution processed by a trained processing model to generate a high-resolution image, and its coordinates are mapped back to the actual physical coordinate system of the touch screen. The output of the tablet computer is then adjusted to improve the operation accuracy.
It improves the precision and interactive experience of users on the touchscreen, enabling better understanding and execution of user intentions, and achieving smoother line drawing or more precise control.
Smart Images
Figure CN120013759B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical field of tablet computers, and in particular to a tablet computer control method and system based on image super-resolution technology. Background Technology
[0002] With the development of mobile devices, tablet computers have become widely used due to their portability and the convenience of touch operation. However, in certain application scenarios, such as professional drawing and precision control, existing touchscreen technology cannot provide sufficient resolution to meet users' high-precision needs. Furthermore, due to limitations in screen size and the physical constraints of display resolution, users may encounter difficulties when performing precise operations. Summary of the Invention
[0003] To at least partially solve the above-mentioned technical problems, this application provides a tablet computer control method and system based on image super-resolution technology.
[0004] Firstly, the tablet computer control method based on image super-resolution technology provided in this application adopts the following technical solution.
[0005] A tablet computer control method based on image super-resolution technology includes:
[0006] Acquire the raw image generated when the user operates the touchscreen of the tablet computer; the raw image is denoted as the first resolution image; the raw image contains a snapshot of the touch trajectory;
[0007] The original image is super-resolution processed by the trained processing model to obtain a second resolution image; the resolution of the second resolution image is greater than that of the first resolution image.
[0008] The coordinates of the super-resolution processed second-resolution image are mapped back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch trajectory on the touch screen can correspond to the coordinate information under the second resolution.
[0009] Adjusting the tablet's output based on the mapped coordinate information makes the final operation result closer to the user's expectations.
[0010] Optionally, the construction of the processing model includes:
[0011] A generator network is used to generate a high-resolution image from a low-resolution input image.
[0012] Discriminator network; the discriminator network is used to distinguish between real high-resolution images and high-resolution images generated by the generator;
[0013] Loss function module; the loss function module is used to quantify the difference between the generated image and the real image, and to guide network optimization;
[0014] The generator network includes:
[0015] Multiple convolutional layers; used to extract image features;
[0016] Upsampling layer; used to increase the spatial size of an image;
[0017] Residual blocks are used to mitigate gradient vanishing and enhance feature learning capabilities.
[0018] Skip connection module; used to maintain information transfer of low-level features during the upsampling process;
[0019] The discriminator network includes:
[0020] Multiple convolutional layers; used to extract image features;
[0021] Downsampling layer; used to reduce the spatial size of an image;
[0022] Fully connected layer; used to ultimately determine the authenticity of the generated image;
[0023] The loss function module includes:
[0024] Content loss module; used to measure pixel-level similarity between generated and real images;
[0025] Adversarial loss module; used to guide the generator to produce more realistic images;
[0026] Edge loss module; used to enhance the edge sharpness of the generated image.
[0027] Optionally, the training of the processing model includes:
[0028] Collect several low-resolution images and their corresponding high-resolution images as a training set;
[0029] Randomly initialize the weights of the generator and discriminator networks;
[0030] In each training iteration:
[0031] Generate high-resolution images using the current generator network;
[0032] Update the discriminator network to minimize its misclassification of real and fake images;
[0033] The generator network is updated to minimize the loss function between the generated and real images and maximize the discriminator network's misclassification of the generated images.
[0034] Repeat the training iterations until the model performance reaches the preset requirements to obtain the processed model.
[0035] Optionally, high-resolution images can be generated using the current generator network, including:
[0036] Obtain a low-resolution image as input;
[0037] The low-resolution image is input into a pre-trained generator model, which is configured to learn the mapping relationship from low-resolution to high-resolution images.
[0038] A series of convolution operations are performed within the generator model, each of which includes weight application, activation function calculation, and normalization.
[0039] After completing all convolutional layer processing, a preliminary high-resolution image is obtained;
[0040] A high-resolution image is obtained by increasing the details of the initial high-resolution image through bilinear interpolation.
[0041] Optionally, updating the discriminator network and generator network includes:
[0042] S501. Extract several real images x from the dataset and use generator G to generate the corresponding images x';
[0043] S502. Input the real image x and the generated image x' into the discriminator network respectively to obtain the discrimination results Dr and Dg, where Dr is the output of the real image x and Dg is the output of the generated image x'.
[0044] S503, Calculate the first loss function of the discriminator network;
[0045] S504. Update the parameters of the discriminator network using the backpropagation algorithm and gradient descent method to minimize the first loss value of the discriminator network.
[0046] S505. Generate a new batch of random noise z, and use the updated generator to generate a new image x”;
[0047] S506. Calculate the second loss function of the updated generator;
[0048] S507. Update the parameters of the generator network using the backpropagation algorithm and gradient descent method to minimize the generator's second loss.
[0049] Repeat steps S501 through S507 until the predetermined number of training rounds is reached.
[0050] Optionally, the method further includes acquiring the raw image generated when the user operates the touchscreen of the tablet computer, and then further comprising:
[0051] Analyze user interaction data to identify the image areas that users interact with most frequently as interactive areas, and denote the remaining areas as non-interactive areas.
[0052] Perform super-resolution processing on the original image in the interactive area;
[0053] The image processed by the super-resolution algorithm is stitched together with the image of the non-interactive area to form a complete second-resolution image.
[0054] Secondly, the tablet computer control system based on image super-resolution technology provided in this application adopts the following technical solution.
[0055] A tablet computer control system based on image super-resolution technology includes:
[0056] The first processing module is configured to: acquire the original image generated when the user operates the touch screen of the tablet computer; the original image is denoted as the first resolution image; the original image contains a snapshot of the touch trajectory;
[0057] The second processing module is used to: perform super-resolution processing on the original image using a trained processing model to obtain a second resolution image; the resolution of the second resolution image is greater than that of the first resolution image;
[0058] The third processing module is used to: map the coordinates of the super-resolution processed second-resolution image back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch trajectory on the touch screen can correspond to the coordinate information under the second resolution.
[0059] The fourth processing module is used to adjust the tablet computer's output based on the mapped coordinate information so that the final operation result is closer to the user's expectations.
[0060] Thirdly, this application discloses an electronic device including a memory and a processor, wherein the memory stores a computer program that is loaded by the processor and executes any of the methods described above.
[0061] Fourthly, this application discloses a computer-readable storage medium storing a computer program that can be loaded by a processor and execute any of the methods described above. Attached Figure Description
[0062] Figure 1 This is a flowchart of a tablet computer control method based on image super-resolution technology according to an embodiment of this application;
[0063] Figure 2This is a system block diagram of a tablet computer control method based on image super-resolution technology according to an embodiment of this application;
[0064] In the diagram, 201 is the first processing module; 202 is the second processing module; 203 is the third processing module; and 204 is the fourth processing module. Detailed Implementation
[0065] The following is in conjunction with the appendix Figure 1-2 The present application will be further described with reference to specific embodiments:
[0066] First, it should be noted that in the description of this application, the use of directional terms such as "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer" indicates the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for descriptive purposes and do not indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the use of numerical quantifiers such as "first," "second," and "third" is for descriptive purposes only and should not be construed as indicating or implying relative importance. Additionally, in this application, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, interference fits, transition fits, or integral connections; they can refer to direct connections or indirect connections through an intermediate medium. Therefore, those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.
[0067] This application discloses a tablet computer control method based on image super-resolution technology. (Refer to...) Figure 1 As one implementation of a tablet computer control method based on image super-resolution technology, the method includes the following steps:
[0068] Step 101: Acquire the original image generated when the user operates the touch screen of the tablet computer; the original image is denoted as the first resolution image; the original image contains a snapshot of the touch trajectory.
[0069] Specifically, the tablet computer uses sensors to capture user touch actions and generate corresponding images. These images can be snapshots of touch trajectories or location information of touch points.
[0070] Step 102: Perform super-resolution processing on the original image using the trained processing model to obtain a second resolution image; the resolution of the second resolution image is greater than that of the first resolution image.
[0071] Specifically, by using a pre-trained super-resolution processing model, low-resolution touch trajectory snapshots can be transformed into higher-resolution images.
[0072] Step 103: Map the coordinates of the super-resolution processed second-resolution image back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch trajectory on the touch screen can correspond to the coordinate information under the second resolution.
[0073] Specifically, each pixel in the enhanced image is mapped to its actual physical location on the touchscreen. This mapping allows the system to more accurately recognize user touch actions and leverages the additional details in the high-resolution image to improve its understanding of user intent.
[0074] Step 104: Adjust the tablet computer's output based on the mapped coordinate information to make the final operation result closer to the user's expectations.
[0075] Specifically, the tablet's output is adjusted based on high-precision coordinate information, such as generating smoother lines in drawing applications or achieving more precise control in games. In this way, the user's interactive experience is significantly improved, and the system is now better able to understand and execute the user's intentions.
[0076] As a specific implementation of a tablet computer control method based on image super-resolution technology, the construction of the processing model includes:
[0077] A generator network is used to generate a high-resolution image from a low-resolution input image.
[0078] Discriminator network; the discriminator network is used to distinguish between real high-resolution images and high-resolution images generated by the generator;
[0079] Loss function module; the loss function module is used to quantify the difference between the generated image and the real image, and to guide network optimization;
[0080] The generator network includes:
[0081] Multiple convolutional layers; used to extract image features;
[0082] Upsampling layer; used to increase the spatial size of an image;
[0083] Residual blocks are used to mitigate gradient vanishing and enhance feature learning capabilities.
[0084] Skip connection module; used to maintain information transfer of low-level features during the upsampling process;
[0085] The discriminator network includes:
[0086] Multiple convolutional layers; used to extract image features;
[0087] Downsampling layer; used to reduce the spatial size of an image;
[0088] Fully connected layer; used to ultimately determine the authenticity of the generated image;
[0089] The loss function module includes:
[0090] Content loss module; used to measure pixel-level similarity between generated and real images;
[0091] Adversarial loss module; used to guide the generator to produce more realistic images;
[0092] Edge loss module; used to enhance the edge sharpness of the generated image.
[0093] Specifically, convolutional layers capture local features in the input image, such as edges and textures. They scan the image using filters to extract useful features. Upsampling layers enlarge low-resolution images to the desired high resolution by increasing their spatial dimensions. Common upsampling techniques include nearest-neighbor interpolation and bilinear interpolation. Residual blocks, by introducing skip connections, allow information and gradients to be more easily passed through deep networks, helping the model learn more complex patterns and preventing the vanishing gradient problem. Skip connections help preserve and pass low-level features to higher-level parts of the generated image, ensuring that the generated image is not only visually realistic but also rich in detail. Convolutional layers in the discriminator network are also used for feature extraction. Downsampling layers reduce the spatial dimension of the image and decrease computational complexity. Fully connected layers act as decision layers, making judgments based on previously extracted features to determine whether the input image is realistic. Content loss ensures that the generated image is as consistent as possible with the real image at the pixel level, achieved by comparing the differences between the two. Adversarial loss encourages the generator network to generate images that deceive the discriminator network, making it unable to distinguish between generated and real images. Edge loss is specifically designed to enhance edge sharpness in generated images.
[0094] As a specific implementation of a tablet computer control method based on image super-resolution technology, the training of the processing model includes:
[0095] Collect several low-resolution images and their corresponding high-resolution images as a training set;
[0096] Randomly initialize the weights of the generator and discriminator networks;
[0097] In each training iteration:
[0098] Generate high-resolution images using the current generator network;
[0099] Update the discriminator network to minimize its misclassification of real and fake images;
[0100] The generator network is updated to minimize the loss function between the generated and real images and maximize the discriminator network's misclassification of the generated images.
[0101] Repeat the training iterations until the model performance reaches the preset requirements to obtain the processed model.
[0102] Specifically,
[0103] As a specific implementation of a tablet computer control method based on image super-resolution technology, high-resolution images are generated using current generator networks, including:
[0104] Obtain a low-resolution image as input;
[0105] The low-resolution image is input into a pre-trained generator model, which is configured to learn the mapping relationship from low-resolution to high-resolution images.
[0106] A series of convolution operations are performed within the generator model, each of which includes weight application, activation function calculation, and normalization.
[0107] After completing all convolutional layer processing, a preliminary high-resolution image is obtained;
[0108] A high-resolution image is obtained by increasing the details of the initial high-resolution image through bilinear interpolation.
[0109] Specifically, the generator and discriminator networks are randomly initialized, and these two networks are alternately optimized in each iteration: first, the generator is used to improve the image resolution, then the discriminator is updated to better distinguish between real and synthetic images, and then the generator is optimized to generate high-resolution images that are more difficult for the discriminator to distinguish. This process is repeated until the model performance meets the predetermined standard, thereby realizing a model that can generate highly realistic and detailed high-resolution images.
[0110] As one implementation method of tablet computer control based on image super-resolution technology, updating the discriminator network and generator network includes:
[0111] S501. Extract several real images x from the dataset and use generator G to generate the corresponding images x';
[0112] S502. Input the real image x and the generated image x' into the discriminator network respectively to obtain the discrimination results Dr and Dg, where Dr is the output of the real image x and Dg is the output of the generated image x'.
[0113] S503, Calculate the first loss function of the discriminator network;
[0114] S504. Update the parameters of the discriminator network using the backpropagation algorithm and gradient descent method to minimize the first loss value of the discriminator network.
[0115] S505. Generate a new batch of random noise z, and use the updated generator to generate a new image x”;
[0116] S506. Calculate the second loss function of the updated generator;
[0117] S507. Update the parameters of the generator network using the backpropagation algorithm and gradient descent method to minimize the generator's second loss.
[0118] Repeat steps S501 through S507 until the predetermined number of training rounds is reached.
[0119] Specifically, real images are extracted from the dataset, and high-resolution images are generated using a generator. These two types of images are then fed into a discriminator network to evaluate their realism. The discriminator's first loss function is calculated to measure its accuracy, and the discriminator parameters are adjusted accordingly to improve its discrimination ability. Next, new image samples are generated, and the generator's second loss function is calculated to evaluate the quality of the generated images. The generator parameters are then updated to optimize the realism and detail of the generated images. This iterative process optimizes the adversarial relationship between the generator and the discriminator.
[0120] As one implementation of a tablet computer control method based on image super-resolution technology, the method acquires the original image generated when the user operates the tablet computer's touchscreen, and then further includes:
[0121] Analyze user interaction data to identify the image areas that users interact with most frequently as interactive areas, and denote the remaining areas as non-interactive areas.
[0122] Perform super-resolution processing on the original image in the interactive area;
[0123] The image processed by the super-resolution algorithm is stitched together with the image of the non-interactive area to form a complete second-resolution image.
[0124] Specifically, fast processing may sacrifice image quality, while high-quality processing requires more time and processor resources. The above scheme uses different processing strategies for different parts of the image, performing high-precision super-resolution processing on key areas, while not performing high-precision super-resolution processing on the background or other unimportant parts.
[0125] This application also discloses a tablet computer control system based on image super-resolution technology, including:
[0126] The first processing module 201 is configured to: acquire an original image generated when a user operates the touchscreen of a tablet computer; the original image is denoted as a first resolution image; the original image contains a snapshot of the touch trajectory;
[0127] The second processing module 202 is used to: perform super-resolution processing on the original image using a trained processing model to obtain a second resolution image; the resolution of the second resolution image is greater than that of the first resolution image;
[0128] The third processing module 203 is used to: map the coordinates of the super-resolution processed second-resolution image back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch trajectory on the touch screen can correspond to the coordinate information under the second resolution.
[0129] The fourth processing module 204 is used to adjust the output of the tablet computer based on the mapped coordinate information so that the final operation result is closer to the user's expectations.
[0130] This application also discloses an electronic device.
[0131] Specifically, the device includes a memory and a processor, the memory storing a computer program that can be loaded by the processor and executed any of the aforementioned tablet computer control methods based on image super-resolution technology.
[0132] This application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed as any of the above-described tablet computer control methods based on image super-resolution technology. The computer-readable storage medium includes, for example, various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0133] It should be noted that the above embodiments are only used to illustrate this application and are not intended to limit the technical solutions described in this application. Although this specification has described this application in detail with reference to the above embodiments, those skilled in the art should understand that they can still make modifications or equivalent substitutions to this application. All technical solutions and improvements that do not depart from the spirit and scope of this application should be covered within the scope of the claims of this application.
Claims
1. A tablet computer control method based on image super-resolution technology, characterized in that, include: Acquire raw images generated when a user interacts with the touchscreen of a tablet. The original image is denoted as the first resolution image; The original image contains a snapshot of the touch trajectory; The original image is super-resolution processed by the trained processing model to obtain a second resolution image; the resolution of the second resolution image is greater than that of the first resolution image. The coordinates of the super-resolution processed second-resolution image are mapped back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch trajectory on the touch screen can correspond to the coordinate information under the second resolution. Adjusting the tablet's output based on the mapped coordinate information makes the final operation result closer to the user's expectations; The construction of the processing model includes: A generator network is used to generate a high-resolution image from a low-resolution input image. Discriminator network; the discriminator network is used to distinguish between real high-resolution images and high-resolution images generated by the generator; Loss function module; the loss function module is used to quantify the difference between the generated image and the real image, and to guide network optimization; The generator network includes: Multiple convolutional layers; used to extract image features; Upsampling layer; used to increase the spatial size of an image; Residual blocks are used to mitigate gradient vanishing and enhance feature learning capabilities. Skip connection module; used to maintain information transfer of low-level features during the upsampling process; The discriminator network includes: Multiple convolutional layers; used to extract image features; Downsampling layer; used to reduce the spatial size of an image; Fully connected layer; used to ultimately determine the authenticity of the generated image; The loss function module includes: Content loss module; used to measure pixel-level similarity between generated and real images; Adversarial loss module; used to guide the generator to produce more realistic images; Edge loss module; used to enhance the edge sharpness of the generated image; The training of the processing model includes: Collect several low-resolution images and their corresponding high-resolution images as a training set; Randomly initialize the weights of the generator and discriminator networks; In each training iteration: Generate high-resolution images using the current generator network; Update the discriminator network to minimize its misclassification of real and fake images; The generator network is updated to minimize the loss function between the generated and real images and maximize the discriminator network's misclassification of the generated images. Repeat the training iterations until the model performance reaches the preset requirements to obtain the processed model.
2. The tablet computer control method based on image super-resolution technology according to claim 1, characterized in that, Generate high-resolution images using the current generator network, including: Obtain a low-resolution image as input; The low-resolution image is input into a pre-trained generator model, which is configured to learn the mapping relationship from low-resolution to high-resolution images. A series of convolution operations are performed within the generator model, each of which includes weight application, activation function calculation, and normalization. After completing all convolutional layer processing, a preliminary high-resolution image is obtained; A high-resolution image is obtained by increasing the details of the initial high-resolution image through bilinear interpolation.
3. The tablet computer control method based on image super-resolution technology according to claim 2, characterized in that, Updating the discriminator network and generator network includes: S501. Extract several real images x from the dataset and use generator G to generate the corresponding images x'; S502. Input the real image x and the generated image x' into the discriminator network respectively to obtain the discrimination results Dr and Dg, where Dr is the output of the real image x and Dg is the output of the generated image x'. S503, Calculate the first loss function of the discriminator network; S504. Update the parameters of the discriminator network using the backpropagation algorithm and gradient descent method to minimize the first loss value of the discriminator network. S505. Generate a new batch of random noise z, and use the updated generator to generate a new image x''; S506. Calculate the second loss function of the updated generator; S507. Update the parameters of the generator network using the backpropagation algorithm and gradient descent method to minimize the generator's second loss. Repeat steps S501 through S507 until the predetermined number of training rounds is reached.
4. The tablet computer control method based on image super-resolution technology according to claim 3, characterized in that, The method further includes acquiring the raw image generated when a user operates the touchscreen of a tablet computer, and then further comprising: Analyze user interaction data to identify the image areas that users interact with most frequently as interactive areas, and denote the remaining areas as non-interactive areas. Perform super-resolution processing on the original image in the interactive area; The image processed by the super-resolution algorithm is stitched together with the image of the non-interactive area to form a complete second-resolution image.
5. A tablet computer control system based on image super-resolution technology, wherein the tablet computer control system based on image super-resolution technology is used to execute the tablet computer control method based on image super-resolution technology according to any one of claims 1 to 4, characterized in that, include: The first processing module is configured to: acquire the original image generated when the user operates the touch screen of the tablet computer; the original image is denoted as the first resolution image; the original image contains a snapshot of the touch trajectory; The second processing module is used to: perform super-resolution processing on the original image using a trained processing model to obtain a second resolution image; the resolution of the second resolution image is greater than that of the first resolution image; The third processing module is used to: map the coordinates of the super-resolution processed second-resolution image back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch trajectory on the touch screen can correspond to the coordinate information under the second resolution. The fourth processing module is used to adjust the tablet computer's output based on the mapped coordinate information.
6. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that is loaded and executed by the processor according to any one of claims 1 to 4.
7. A computer-readable storage medium, characterized in that, It stores a computer program that can be loaded by a processor and executed according to any one of claims 1 to 4.