Tablet computer control method and system based on image super-division technology

Abstract

The invention relates to a tablet personal computer control method and system based on an image super-division technology, and relates to the technical field of tablet personal computers, and the method comprises the following steps: obtaining an original image generated when a user operates a touch screen of a tablet personal computer; the original image is recorded as a first resolution image; the original image comprises a snapshot of a touch track; performing super-resolution processing on the original image through a trained processing model to obtain a second resolution image; the resolution of the second resolution image is greater than that of the first resolution image; mapping coordinates of the second resolution image after super-resolution processing back to an actual physical coordinate system of the touch screen, so that each touch point of a touch track of a user on the touch screen can correspond to coordinate information under the second resolution; the output of the tablet computer is adjusted according to the mapped coordinate information, so that the final operation result is closer to the expectation of the user.

Description

Technical Field

[0001] The present application relates to the technical field of tablet computers, and in particular to a tablet computer control method and system based on image super-resolution technology. Background Art

[0002] With the development of mobile devices, tablet computers are widely used due to their portability and the convenience of touch operation. However, in some application scenarios, such as professional drawing and precision control, the existing touch screen technology cannot provide sufficient resolution to meet the high-precision needs of users. In addition, due to the physical limitations of screen size and display resolution, users may encounter difficulties when performing fine operations. Summary of the invention

[0003] In order to at least partially solve the above technical problems, the present application provides a tablet computer control method and system based on image super-resolution technology.

[0004] In a first aspect, the present application provides a tablet computer control method based on image super-resolution technology, which adopts the following technical solution.

[0005] A tablet computer control method based on image super-resolution technology, comprising:

[0006] Acquire an original image generated when a user operates a touch screen of a tablet computer; the original image is recorded as a first resolution image; the original image includes a snapshot of the touch track;

[0007] Performing super-resolution processing on the original image through the trained processing model to obtain a second-resolution image; the resolution of the second-resolution image is greater than that of the first-resolution image;

[0008] Mapping the coordinates of the second resolution image after super-resolution processing back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch track on the touch screen can correspond to the coordinate information under the second resolution;

[0009] The output of the tablet computer is adjusted according to the mapped coordinate information so that the final operation result is closer to the user's expectations.

[0010] Optionally, the construction of the processing model includes:

[0011] A generator network; the generator network is used to generate a high-resolution image from a low-resolution input image;

[0012] A discriminator network; the discriminator network is used to distinguish between a real high-resolution image and a high-resolution image generated by a generator;

[0013] A loss function module; the loss function module is used to quantify the difference between the generated image and the real image and guide network optimization;

[0014] Wherein, the generator network comprises:

[0015] Multiple convolutional layers; used to extract image features;

[0016] Upsampling layer; used to increase the spatial size of the image;

[0017] Residual block; used to alleviate gradient disappearance and enhance feature learning ability;

[0018] Skip connection module; used to keep the information transfer of low-level features during upsampling;

[0019] Wherein, the discriminator network comprises:

[0020] Multiple convolutional layers; used to extract image features;

[0021] Downsampling layer; used to reduce the spatial size of the image;

[0022] Fully connected layer; used to finally judge the authenticity of the generated image;

[0023] Among them, the loss function module includes:

[0024] Content loss module; used to measure the pixel-level similarity between the generated image and the real image;

[0025] Adversarial loss module; used to guide the generator to generate more realistic images;

[0026] Edge loss module; used to enhance the edge clarity of the generated image.

[0027] Optionally, the training of the processing model includes:

[0028] Collect several low-resolution images and their corresponding high-resolution images as training sets;

[0029] Randomly initialize the weights of the generator and discriminator networks;

[0030] In each training iteration:

[0031] Generate high-resolution images using the current generator network;

[0032] Update the discriminator network to minimize the discriminator network's misclassification of true and false images;

[0033] Update the generator network to minimize the loss function between the generated image and the real image and maximize the misclassification of the discriminator network for the generated image;

[0034] The training iterations are repeatedly performed until the model performance meets the preset requirements to obtain the processing model.

[0035] Optionally, generate high-resolution images using the current generator network, including:

[0036] Get a low-resolution image as input;

[0037] Inputting the low-resolution image into a pre-trained generator model, wherein the generator model is configured to learn a mapping relationship from the low-resolution to the high-resolution image;

[0038] Performing a series of convolution operations within the generator model, each of which includes weight application, activation function calculation, and normalization;

[0039] After completing all convolutional layers, a preliminary high-resolution image is obtained;

[0040] The details of the preliminary high-resolution image are increased by bilinear interpolation to obtain a high-resolution image.

[0041] Optionally, updating the discriminator network and the generator network includes:

[0042] S501, extracting several real images x from the data set, and using the generator G to generate corresponding images x';

[0043] S502, input the real image x and the generated image x' into the discriminator network respectively, and obtain the discriminant results Dr and Dg, where Dr is the output of the real image x and Dg is the output of the generated image x';

[0044] S503, calculating the first loss function of the discriminator network;

[0045] S504, using the back propagation algorithm and the gradient descent method to update the parameters of the discriminator network so that the first loss value of the discriminator network is minimized;

[0046] S505, generate a batch of new random noise z, and use the updated generator to generate a new image "x";

[0047] S506, calculating the second loss function of the updated generator;

[0048] S507, using the back propagation algorithm and the gradient descent method to update the parameters of the generator network so as to minimize the second loss of the generator;

[0049] S501 to S507 are repeated until a predetermined number of training rounds is reached.

[0050] Optionally, an original image generated when a user operates a touch screen of a tablet computer is obtained, and then the method further includes:

[0051] Analyze the user interaction behavior data, determine the image area where users interact most frequently as the interactive area, and the rest as the non-interactive area;

[0052] Performing super-resolution processing on the original image in the interaction area;

[0053] The image processed by the super-resolution algorithm is spliced ​​with the image of the non-interaction area to form a complete second-resolution image.

[0054] In a second aspect, the present application provides a tablet computer control system based on image super-resolution technology, which adopts the following technical solution.

[0055] A tablet computer control system based on image super-resolution technology, comprising:

[0056] The first processing module is used to: obtain an original image generated when a user operates the touch screen of the tablet computer; the original image is recorded as a first resolution image; the original image includes a snapshot of the touch track;

[0057] A second processing module is used to: perform super-resolution processing on the original image through a trained processing model to obtain a second-resolution image; the resolution of the second-resolution image is greater than that of the first-resolution image;

[0058] A third processing module is used to map the coordinates of the second resolution image after super-resolution processing back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch track on the touch screen can correspond to the coordinate information under the second resolution;

[0059] The fourth processing module is used to adjust the output of the tablet computer according to the mapped coordinate information so that the final operation result is closer to the user's expectation.

[0060] In a third aspect, the present application discloses an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that is loaded by the processor and executes any of the above methods.

[0061] In a fourth aspect, the present application discloses a computer-readable storage medium storing a computer program that can be loaded by a processor and execute any of the above methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is a flow chart of a tablet computer control method based on image super-resolution technology according to an embodiment of the present application;

[0063] Figure 2This is a system block diagram of a tablet computer control method based on image super-resolution technology in an embodiment of the present application;

[0064] In the figure, 201 is a first processing module; 202 is a second processing module; 203 is a third processing module; 204 is a fourth processing module. DETAILED DESCRIPTION

[0065] The following is combined with Figure 1-2 The present application is further described with specific embodiments:

[0066] First of all, it should be noted that in the description of this application, if the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside" and other directional words appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, which is only for the convenience of description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of this application; in addition, if the terms "first", "second", "third" and other numerical quantifiers appear, they are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in this application, unless otherwise clearly specified and limited, if the terms "installed", "connected", and "connected" appear, they should be understood in a broad sense, for example, it can be a fixed connection, or a detachable connection, a limited connection such as an interference fit, a transition fit, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium; therefore, for ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0067] The present application embodiment discloses a tablet computer control method based on image super-resolution technology. Figure 1 As an implementation of a tablet computer control method based on image super-resolution technology, a tablet computer control method based on image super-resolution technology includes the following steps:

[0068] Step 101: Acquire an original image generated when a user operates a touch screen of a tablet computer; the original image is recorded as a first resolution image; the original image includes a snapshot of the touch trajectory.

[0069] Specifically, the sensor equipped by the tablet computer can capture the user's touch operation and generate corresponding images, which can be snapshots of the touch track or the location information of the touch point.

[0070] Step 102: Perform super-resolution processing on the original image using the trained processing model to obtain a second-resolution image; the resolution of the second-resolution image is greater than that of the first-resolution image.

[0071] Specifically, by using a pre-trained super-resolution processing model, low-resolution touch trajectory snapshots can be converted into higher-resolution images.

[0072] Step 103: Map the coordinates of the second resolution image after super-resolution processing back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch trajectory on the touch screen can correspond to the coordinate information under the second resolution.

[0073] Specifically, each pixel in the enhanced image is mapped to its actual physical location on the touch screen. Through this mapping, the system can more accurately identify the user's touch actions and can use the additional details in the high-resolution image to improve the understanding of the user's intentions.

[0074] Step 104: Adjust the output of the tablet computer according to the mapped coordinate information so that the final operation result is closer to the user's expectation.

[0075] Specifically, the tablet's output is adjusted based on high-precision coordinate information, such as generating smoother lines in drawing applications or achieving more precise control in games. In this way, the user's interactive experience is significantly improved, and the system is now able to better understand and execute the user's intentions.

[0076] As a specific implementation of a tablet computer control method based on image super-resolution technology, the construction of the processing model includes:

[0077] A generator network; the generator network is used to generate a high-resolution image from a low-resolution input image;

[0078] A discriminator network; the discriminator network is used to distinguish between a real high-resolution image and a high-resolution image generated by a generator;

[0079] A loss function module; the loss function module is used to quantify the difference between the generated image and the real image and guide network optimization;

[0080] Wherein, the generator network comprises:

[0081] Multiple convolutional layers; used to extract image features;

[0082] Upsampling layer; used to increase the spatial size of the image;

[0083] Residual block; used to alleviate gradient disappearance and enhance feature learning ability;

[0084] Skip connection module; used to keep the information transfer of low-level features during upsampling;

[0085] Wherein, the discriminator network comprises:

[0086] Multiple convolutional layers; used to extract image features;

[0087] Downsampling layer; used to reduce the spatial size of the image;

[0088] Fully connected layer; used to finally judge the authenticity of the generated image;

[0089] Among them, the loss function module includes:

[0090] Content loss module; used to measure the pixel-level similarity between the generated image and the real image;

[0091] Adversarial loss module; used to guide the generator to generate more realistic images;

[0092] Edge loss module; used to enhance the edge clarity of the generated image.

[0093] Specifically, the convolutional layer can capture local features in the input image, such as edges, textures, etc. The convolutional layer scans the image through filters and extracts useful features. The upsampling layer enlarges the low-resolution image to the required high resolution by increasing the spatial size of the image. Common upsampling techniques include nearest neighbor interpolation and bilinear interpolation. The residual block allows information and gradients to be more easily transferred in the deep network by introducing skip connections, thereby helping the model learn more complex patterns and preventing the gradient vanishing problem. Skip connections help retain and transfer low-level features to the high-level parts of the generated image, ensuring that the generated image is not only visually realistic but also rich in details. The convolutional layer in the discriminator network is also used for feature extraction. The downsampling layer reduces the spatial dimension of the image and reduces the computational complexity. The fully connected layer acts as a decision layer, making a judgment based on the previously extracted features to determine whether the input image is real. The content loss ensures that the generated image is as consistent as possible with the real image at the pixel level, which is achieved by comparing the differences between the two. The adversarial loss encourages the generator network to generate images that fool the discriminator network, making it unable to distinguish between generated images and real images. The edge loss is specifically used to enforce edge clarity in the generated images.

[0094] As a specific implementation of a tablet computer control method based on image super-resolution technology, the training of the processing model includes:

[0095] Collect several low-resolution images and their corresponding high-resolution images as training sets;

[0096] Randomly initialize the weights of the generator and discriminator networks;

[0097] In each training iteration:

[0098] Generate high-resolution images using the current generator network;

[0099] Update the discriminator network to minimize the discriminator network's misclassification of true and false images;

[0100] Update the generator network to minimize the loss function between the generated image and the real image and maximize the misclassification of the discriminator network for the generated image;

[0101] The training iterations are repeatedly performed until the model performance meets the preset requirements to obtain the processing model.

[0102] Specifically,

[0103] As a specific implementation of a tablet computer control method based on image super-resolution technology, a high-resolution image is generated using the current generator network, including:

[0104] Get a low-resolution image as input;

[0105] Inputting the low-resolution image into a pre-trained generator model, wherein the generator model is configured to learn a mapping relationship from the low-resolution to the high-resolution image;

[0106] Performing a series of convolution operations within the generator model, each of which includes weight application, activation function calculation, and normalization;

[0107] After completing all convolutional layers, a preliminary high-resolution image is obtained;

[0108] The details of the preliminary high-resolution image are increased by bilinear interpolation to obtain a high-resolution image.

[0109] Specifically, through randomly initialized generator and discriminator networks, the two networks are optimized alternately in each iteration: first, the generator is used to improve the image resolution, then the discriminator is updated to better distinguish between real and synthetic images, and then the generator is optimized to generate high-resolution images that are more difficult for the discriminator to distinguish. This process is repeated until the model performance meets the predetermined standards, thereby realizing a model that can generate highly realistic and detailed high-resolution images.

[0110] As one implementation of a tablet computer control method based on image super-resolution technology, updating the discriminator network and the generator network includes:

[0111] S501, extracting several real images x from the data set, and using the generator G to generate corresponding images x';

[0112] S502, input the real image x and the generated image x' into the discriminator network respectively, and obtain the discriminant results Dr and Dg, where Dr is the output of the real image x and Dg is the output of the generated image x';

[0113] S503, calculating the first loss function of the discriminator network;

[0114] S504, using the back propagation algorithm and the gradient descent method to update the parameters of the discriminator network so that the first loss value of the discriminator network is minimized;

[0115] S505, generate a batch of new random noise z, and use the updated generator to generate a new image "x";

[0116] S506, calculating the second loss function of the updated generator;

[0117] S507, using the back propagation algorithm and the gradient descent method to update the parameters of the generator network so as to minimize the second loss of the generator;

[0118] S501 to S507 are repeated until a predetermined number of training rounds is reached.

[0119] Specifically, by extracting real images from the data set and using the generator to generate high-resolution images, these two types of images are then fed into the discriminator network to evaluate their authenticity, calculate the first loss function of the discriminator to measure its judgment accuracy, and adjust the discriminator parameters accordingly to improve its discrimination ability. Then, new image samples are generated, the second loss function of the generator is calculated to evaluate the quality of the generated image, and the generator parameters are updated to optimize the realism and details of the generated image; the repeated iterations of this process optimize the adversarial nature between the generator and the discriminator.

[0120] As one implementation of a tablet computer control method based on image super-resolution technology, an original image generated when a user operates a touch screen of the tablet computer is obtained, and then the method further includes:

[0121] Analyze the user interaction behavior data, determine the image area where users interact most frequently as the interactive area, and the rest as the non-interactive area;

[0122] Performing super-resolution processing on the original image in the interaction area;

[0123] The image processed by the super-resolution algorithm is spliced ​​with the image of the non-interaction area to form a complete second-resolution image.

[0124] Specifically, fast processing may sacrifice image quality, while high-quality processing requires more time and processor resources. The above scheme adopts different processing strategies for different parts of the image, performs high-precision super-resolution processing on key areas, and does not use high-precision super-resolution processing on the background or other unimportant parts.

[0125] The present application also discloses a tablet computer control system based on image super-resolution technology, including:

[0126] The first processing module 201 is used to: obtain an original image generated when a user operates the touch screen of the tablet computer; the original image is recorded as a first resolution image; the original image includes a snapshot of the touch track;

[0127] The second processing module 202 is used to: perform super-resolution processing on the original image through the trained processing model to obtain a second-resolution image; the resolution of the second-resolution image is greater than that of the first-resolution image;

[0128] The third processing module 203 is used to map the coordinates of the second resolution image after super-resolution processing back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch track on the touch screen can correspond to the coordinate information under the second resolution;

[0129] The fourth processing module 204 is used to adjust the output of the tablet computer according to the mapped coordinate information so that the final operation result is closer to the user's expectation.

[0130] The embodiment of the present application also discloses an electronic device.

[0131] Specifically, the device includes a memory and a processor, and the memory stores a computer program that can be loaded by the processor and execute any one of the above-mentioned tablet computer control methods based on image super-resolution technology.

[0132] The embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and execute any of the above-mentioned tablet computer control methods based on image super-resolution technology, and the computer-readable storage medium includes, for example: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0133] It should be noted that the above embodiments are only used to illustrate the present application and are not intended to limit the technical solutions described in the present application. Although the present application has been described in detail in this specification with reference to the above embodiments, a person of ordinary skill in the art should understand that a person of ordinary skill in the art can still modify or make equivalent substitutions to the present application, and all technical solutions and improvements thereof that do not depart from the spirit and scope of the present application should be included in the scope of the claims of the present application.

Claims

1. A tablet computer control method based on image super-resolution technology, characterized in that: include: Acquire the original image generated when the user operates the touch screen of the tablet computer; The original image is recorded as a first resolution image; The original image contains a snapshot of the touch trajectory; Performing super-resolution processing on the original image through the trained processing model to obtain a second-resolution image; the resolution of the second-resolution image is greater than that of the first-resolution image; Mapping the coordinates of the second resolution image after super-resolution processing back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch track on the touch screen can correspond to the coordinate information under the second resolution; The output of the tablet computer is adjusted according to the mapped coordinate information so that the final operation result is closer to the user's expectations.

2. The tablet computer control method based on image super-resolution technology according to claim 1, characterized in that: The construction of the processing model includes: A generator network; the generator network is used to generate a high-resolution image from a low-resolution input image; A discriminator network; the discriminator network is used to distinguish between a real high-resolution image and a high-resolution image generated by a generator; A loss function module; the loss function module is used to quantify the difference between the generated image and the real image and guide network optimization; Wherein, the generator network comprises: Multiple convolutional layers; used to extract image features; Upsampling layer; used to increase the spatial size of the image; Residual block; used to alleviate gradient disappearance and enhance feature learning ability; Skip connection module; used to keep the information transfer of low-level features during upsampling; Wherein, the discriminator network comprises: Multiple convolutional layers; used to extract image features; Downsampling layer; used to reduce the spatial size of the image; Fully connected layer; used to finally judge the authenticity of the generated image; Among them, the loss function module includes: Content loss module; used to measure the pixel-level similarity between the generated image and the real image; Adversarial loss module; used to guide the generator to generate more realistic images; Edge loss module; used to enhance the edge clarity of the generated image.

3. The tablet computer control method based on image super-resolution technology according to claim 2 is characterized in that: The training of the processing model includes: Collect several low-resolution images and their corresponding high-resolution images as training sets; Randomly initialize the weights of the generator and discriminator networks; In each training iteration: Generate high-resolution images using the current generator network; Update the discriminator network to minimize the discriminator network's misclassification of true and false images; Update the generator network to minimize the loss function between the generated image and the real image and maximize the misclassification of the discriminator network for the generated image; The training iterations are repeatedly performed until the model performance meets the preset requirements to obtain the processing model.

4. The tablet computer control method based on image super-resolution technology according to claim 3 is characterized in that: Generate high-resolution images using the current generator network, including: Get a low-resolution image as input; Inputting the low-resolution image into a pre-trained generator model, wherein the generator model is configured to learn a mapping relationship from the low-resolution to the high-resolution image; Performing a series of convolution operations within the generator model, each of which includes weight application, activation function calculation, and normalization; After completing all convolutional layers, a preliminary high-resolution image is obtained; The details of the preliminary high-resolution image are increased by bilinear interpolation to obtain a high-resolution image.

5. The tablet computer control method based on image super-resolution technology according to claim 4 is characterized in that: Updating the discriminator network and the generator network includes: S501, extracting several real images x from the data set, and using the generator G to generate corresponding images x'; S502, input the real image x and the generated image x' into the discriminator network respectively, and obtain the discriminant results Dr and Dg, where Dr is the output of the real image x and Dg is the output of the generated image x'; S503, calculating the first loss function of the discriminator network; S504, using the back propagation algorithm and the gradient descent method to update the parameters of the discriminator network so that the first loss value of the discriminator network is minimized; S505, generate a new batch of random noise z, and use the updated generator to generate new images" x； S506, calculating the second loss function of the updated generator; S507, using the back propagation algorithm and the gradient descent method to update the parameters of the generator network so as to minimize the second loss of the generator; S501 to S507 are repeated until a predetermined number of training rounds is reached.

6. The tablet computer control method based on image super-resolution technology according to claim 5, characterized in that: Acquire the original image generated when the user operates the touch screen of the tablet computer, and then the method further includes: Analyze the user interaction behavior data, determine the image area where users interact most frequently as the interactive area, and the rest as the non-interactive area; Performing super-resolution processing on the original image in the interaction area; The image processed by the super-resolution algorithm is spliced ​​with the image of the non-interaction area to form a complete second-resolution image.

7. A tablet computer control system based on image super-resolution technology, characterized in that: include: The first processing module is used to: obtain an original image generated when a user operates the touch screen of the tablet computer; the original image is recorded as a first resolution image; the original image includes a snapshot of the touch track; A second processing module is used to: perform super-resolution processing on the original image through a trained processing model to obtain a second-resolution image; the resolution of the second-resolution image is greater than that of the first-resolution image; A third processing module is used to map the coordinates of the second resolution image after super-resolution processing back to the actual physical coordinate system of the touch screen, so that each touch point of the user's touch track on the touch screen can correspond to the coordinate information under the second resolution; The fourth processing module is used to adjust the output of the tablet computer according to the mapped coordinate information so that the final operation result is closer to the user's expectation.

8. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program according to any one of the methods of claims 1 to 6 which is loaded and executed by the processor.

9. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 6.

Images