METHOD FOR PERFORMING A ZOOM OPERATION ON A COMPUTING DEVICE, COMPUTING DEVICE, AND COMPUTER-READABLE MEMORY
The method enhances computing devices' zoom operations by identifying ROI, using CNNs for efficient downscaling and animation, addressing power consumption and image quality issues in existing CNN-based zoom technologies.
Patent Information
- Application Number
- BR112025025159
- Authority / Receiving Office
- BR · BR
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-06-01
- Publication Date
- 2026-11-10
AI Technical Summary
Computing devices face challenges in providing smooth and efficient zoom operations on images due to high power consumption by convolutional neural networks (CNNs) and inadequate image scaling techniques, limiting runtime zoom functions to specific applications.
A method involving a computing device processor that identifies a region of interest (ROI) and uses CNNs to generate a final zoomed-in presentation, downscaling it to intermediate-sized frames, and renders an animation using linear scaling techniques, combining ROI and surrounding image frames for smooth zoom animations.
Enables runtime zoom functions across various applications without significant power consumption, providing smooth animations and maintaining image quality.
Abstract
Description
1 / 50 "METHOD FOR PERFORMING A ZOOM OPERATION ON A COMPUTING DEVICE, COMPUTING DEVICE, AND COMPUTER-READABLE MEMORY" BACKGROUND
[0001] Computing devices, such as smartphones, can utilize Artificial Intelligence (AI) for image processing using a dedicated AI processor or an AI accelerator designed to efficiently execute machine learning algorithms. The AI processor / accelerator can also run convolutional neural networks (CNNs) as part of a CNN-based AI image processing system. CNNs are suitable for various image processing tasks, such as image recognition, object detection, and image segmentation. However, these CNNs perform zoom operations on poorly displayed images, providing zoom animations that are not perceptible. These CNNs also require a lot of power, consuming an excessive amount of battery power. Consequently, on many computing devices, a runtime zoom or runtime magnification operation is only available within certain applications running on the computing device. SUMMARY
[0002] Several aspects include methods for performing a zoom operation on a computing device. Some aspects may include identifying a region of interest (ROI) of an image displayed on a computing device display based on a user zoom command received by the device. Petition 870260046470, dated 05 / 15 / 2026, page 5 / 124 2 / 50 computation, determine a scaling factor for a final zoomed-in presentation of the ROI based on one or both of an image rendering requirement or a frame-generating convolutional neural network (CNN) inference speed, use the CNN to generate the final zoomed-in presentation of the ROI for the determined scaling factor, downscale the final zoomed-in presentation to generate a sequence of intermediate-sized image frames, each having dimensions smaller than the dimensions of the final zoomed-in presentation of the ROI and larger than the dimensions of the ROI, and render a changeover animation at a displayed size of the ROI using the sequence of intermediate image frames of the ROI.
[0003] In some respects, the image rendering requirement may include a number of frames per second (FPS) generated by downscaling the final zoom-in presentation to the generated sequence of intermediate-sized image frames. In some respects, downscaling the final zoom-in presentation to generate the intermediate-sized image frame sequence may be performed using a linear scaling technique. In some respects, determining the scaling factor for the final zoom-in presentation and using CNN to generate the final zoom-in presentation may be performed in response to detecting a user zoom command initialization, and rendering the animation of changes in the displayed size of the ROI using the intermediate image frame sequence may be performed in response to the user zoom command. Petition 870260046470, dated 05 / 15 / 2026, p. 6 / 124 3 / 50
[0004] In some respects, the determination of the scaling factor for the final zoomed-in presentation of the ROI may be additionally based on the dimensions of the ROI and the dimensions of the computing device display. In some respects, the determination of the scaling factor for the final zoomed-in presentation of the ROI may be additionally based on the dimensions of an image element within the ROI and the dimensions of the computing device display. In some respects, generating a sequence of scaled-in image frames of display elements outside the ROI using an interpolation scaling technique. In such respects, rendering the animation of changes in the displayed size of the ROI using the sequence of intermediate image frames of the ROI may include rendering an animation on the display that combines the sequence of intermediate image frames of the ROI and the sequence of enlarged image frames of image elements outside the ROI.
[0005] Additional aspects may include a computing device that has a processor configured with processor-executable instructions to perform various operations corresponding to the methods discussed above. Additional aspects may include a processor-readable non-transient storage medium that stores processor-executable instructions configured to cause a processor to perform various operations corresponding to the method operations discussed above. Additional aspects may include a computing device having multiple means to perform functions corresponding to the method operations discussed above. BRIEF DESCRIPTION OF THE DRAWINGS Petition 870260046470, dated 05 / 15 / 2026, page 7 / 124 4 / 50
[0006] The accompanying drawings, which are incorporated into the present invention and form part of this descriptive report, illustrate exemplary embodiments of the claims and, together with the general description given and the detailed description, serve to explain the attributes of the present invention.
[0007] Figure 1 is a conceptual diagram that illustrates aspects of a method for performing a zoom operation on a computing device according to various modalities.
[0008] Figures 2A and 2B illustrate an example of a neural network 200 suitable for implementation in a computing device according to various modalities.
[0009] Figures 3A and 3B illustrate components of an example functionality that can be included in a convolutional neural network, which can be implemented on a computing device that is configured to implement a generalized framework for continuous learning according to various modalities.
[0010] Figure 4 is a component block diagram illustrating an example computing system implemented as a system in a package (SIP), suitable for implementing various modalities.
[0011] Figure 5A illustrates a method for performing a zoom operation on a computing device according to various modes.
[0012] Figure 5B illustrates operations that can be performed as part of the method for performing a zoom operation on a computing device according to various modalities. Petition 870260046470, dated 05 / 15 / 2026, page 8 / 124 5 / 50
[0013] Figure 6 is a component block diagram illustrating an example of a wireless device suitable for use with multiple modes.
[0014] Figure 7 is a component block diagram illustrating an example wireless communication device suitable for use with various modes.
[0015] Figure 8 illustrates an example of a wearable computing device in the form of a smartwatch suitable for use with various modalities. DETAILED DESCRIPTION
[0016] Various embodiments will be described in detail with reference to the attached drawings. Wherever possible, identical reference numbers will be used in all drawings to refer to identical or related parts. References made to particular examples and implementations are for illustrative purposes only and are not intended to limit the scope of the claims.
[0017] In an overview, various modalities include methods and computing devices configured to implement the methods, to perform a zoom operation on an image or text on a computing device display in response to a user command to zoom in on the image or text. Various modalities may include responding to a user command for a zoom operation (a user command to zoom) by identifying a region of interest (ROI) within the image rendered on a computing device display and determining a final size or zoom magnification factor for the identified ROI. The zoom operation may include the use of CNN image processing to generate a Petition 870260046470, dated 05 / 15 / 2026, page 9 / 124 6 / 50 zoomed-in image of the ROI at final size or zoom factor, generating a sequence of downscaled images that are smaller versions of the zoomed-in ROI image using linear scaling operations, and then producing a zoom animation by sequentially rendering the downscaled images starting with the original ROI, continuing with increasingly larger images, and ending with the zoomed-in ROI image at final size or zoom factor. In this way, the zoomed-in ROI image at final size or zoom factor can display the good resolution enabled through the use of CNN image processing, providing a smooth animation without the power consumption that would be required to generate all the images in the animation using CNN processing.
[0018] The term computing device may be used in the present invention to refer to any or all quantum computing devices, edge computing devices, Internet access gateways, modems, routers, network switches, residential gateways, access points, integrated access devices (IADs), mobile convergence products, network adapters, multiplexers, personal computers, laptop computers, tablet computers, user equipment (UEs), smartphones, personal or mobile multimedia players, personal data assistants (PDAs), palm-top computers, wireless email receivers, multimedia Internet-enabled mobile phones, gaming systems (e.g., Petition 870260046470, dated 05 / 15 / 2026, page 10 / 124 7 / 50 PlayStation™, Xbox™, Nintendo Switch™, etc.), wearable devices (e.g., smartwatches, head-mounted displays, fitness trackers, etc.), media players (e.g., DVD players, ROKU™, AppleTV™, etc.), digital video recorders (DVRs), automotive displays, portable projectors, 3D holographic displays, and other similar devices that include a display and a programmable processor that can be configured to provide multimodal functionality.
[0019] The term system-on-chip (SoC) is used in the present invention to refer to a single integrated circuit (IC) chip containing multiple independent features and / or processors integrated onto a single substrate. A single SoC may contain circuitry for digital, analog, mixed-signal, and radio frequency functions. A single SoC may also include any number of specialized or general-purpose processors (network processors, digital signal processors, modem processors, video processors, etc.), memory blocks (e.g., ROM, RAM, flash, etc.), and features (e.g., timers, voltage regulators, oscillators, etc.).For example, a SoC might include an application processor that operates as the SoC's main processor, a central processing unit (CPU), a microprocessor unit (MPU), an arithmetic logic unit (ALU), etc. SoCs might also include software to control the integrated resources and processors, as well as to control peripheral devices. Petition 870260046470, dated 05 / 15 / 2026, page 11 / 124 8 / 50
[0020] The term system in a package (SIP) can be used in the present invention to refer to a single module or package containing multiple resources, computing units, cores, or processors on two or more IC chips, substrates, or SoCs. For example, a SIP may include a single substrate on which multiple IC chips or semiconductor wafers are stacked in a vertical configuration. Similarly, the SIP may include one or more multi-chip modules (MCMs) in which multiple ICs or semiconductor wafers are packaged on a unifying substrate. A SIP may also include multiple independent SoCs coupled together via a set of high-speed communication circuits and packaged in close proximity, such as on a single motherboard, a single UE, or a single CPU. The proximity of the SoCs facilitates high-speed communications and the sharing of memory and resources.
[0021] The term neural network is used in the present invention to refer to an interconnected group of processing nodes (e.g., neuron models, etc.) that collectively operate as a software application or process that controls a function of a computing device or generates a neural network inference. Individual nodes in a neural network may attempt to emulate biological neurons by receiving input data, performing simple operations on the input data to generate output data, and passing the output data (also called activation) to the next node in the network. Each node may be associated with a weight value that defines Petition 870260046470, dated 05 / 15 / 2026, page 12 / 124 9 / 50 or governs the relationship between input data and activation. Weight values can be determined during a training phase and updated iteratively as data flows through the neural network.
[0022] Deep neural networks implement a layered architecture in which the activation of a first layer of nodes becomes an input for a second layer of nodes, the activation of a second layer of nodes becomes an input for a third layer of nodes, and so on. In this way, computations in a deep neural network can be distributed across a population of processing nodes that make up a computational chain. Deep neural networks may also include activation functions and subfunctions (e.g., a rectified linear unit that eliminates activations below zero, etc.) between the layers. The first layer of nodes in a deep neural network can be called an input layer. The final layer of nodes can be referred to as an output layer. The layers between the input layer and the final layer can be called intermediate layers, hidden layers, or black-box layers.Each layer in a neural network can have multiple inputs and therefore multiple previous or preceding layers. Put another way, multiple layers can be included within a single layer. For ease of reference, some of the modalities are described with reference to a single input or single preceding layer. However, it should be understood that the operations disclosed and described in this application can be applied to each of the multiple inputs in a layer, as well as multiple preceding layers.
[0023] The term convolutional neural network Petition 870260046470, dated 05 / 15 / 2026, page 13 / 124 10 / 50 (CNN) is used in the present invention to refer to a deep neural network in which computation in at least one layer is structured as a convolution. A convolutional neural network may also include multiple convolution-based layers, allowing the neural network to employ a very deep hierarchy of layers. In convolutional neural networks, the weighted sum for each output activation is calculated based on a batch of inputs, and the same weight matrices (called filters) are applied to each output. These networks may also implement a fixed feedforward structure in which all processing nodes comprising a computational chain are used to process each task, regardless of the inputs. In such feedforward neural networks, all calculations are performed as a sequence of operations on the outputs of a previous layer.The final set of operations generates the overall inference result of the neural network, such as a probability that an image contains a specific object (e.g., a person, cat, clock, edge, etc.) or information indicating that a proposed action should be taken.
[0024] The term inference can be used in the present invention to refer to a process that is performed at runtime or during the execution of the software application program corresponding to the neural network. Inference may include traversing the processing nodes in the neural network along a direct path to produce one or more values as an overall activation or overall inference result.
[0025] A computing device can Petition 870260046470, dated 05 / 15 / 2026, page 14 / 124 11 / 50 include display and graphics hardware or subsystems that utilize Artificial Intelligence (AI) to enhance the performance and capabilities of display and graphics processing. In a computing device that uses AI for image processing, the hardware may include a dedicated AI processor or AI accelerator that is designed to execute machine learning algorithms more efficiently than other processors. CNNs are one of the most commonly used AI methods for image processing.
[0026] In a computing device with a CNN-based AI image processing system, the CNNs can be run on the AI processor or accelerator to analyze and process images that are to be displayed on the computing device's screen. The use of AI image processing systems can improve the performance and efficiency of the computing device's display and graphics. AI image processing systems can also enhance a user's experience by supporting attributes such as improved image quality, real-time image enhancement, and more realistic virtual and augmented reality experiences. For example, an AI image processing system can be used to enhance the realism of virtual and augmented reality experiences by improving 3D graphics rendering and object tracking.
[0027] Bilinear and bicubic scalars are interpolation scaling techniques that can be used to resize or scale images. For example, bilinear or bicubic scalars can be used to Petition 870260046470, dated 05 / 15 / 2026, page 15 / 124 12 / 50 scaling is the process of resize an image by increasing or decreasing the number of pixels it contains. Bilinear scaling can involve using linear interpolation to determine the value of new pixels based on the values of nearby pixels in the original image. Bilinear scaling is particularly useful when the original image is smooth and doesn't contain many fine details. Bilinear scaling can cause loss of detail and blurring when used to enlarge images with high levels of detail or texture. Bicubic scaling is a more complex technique that uses cubic interpolation to determine the values of new pixels based on the values of nearby pixels in the original image. Bicubic scaling can produce better results than bilinear scaling, particularly when scaling images with a wide range of image types, including those with high levels of detail or texture.Bicubic scheduling can be slower or more processor-intensive than bilinear scheduling. Both bilinear and bicubic scheduling techniques can be used for image zoom magnification, upscaling, and downscaling.
[0028] AI superresolution is a technique that can be used to increase the resolution of an image or video. AI superresolution is commonly used in image and video processing tasks to improve the visual quality of the image or video. AI superresolution can be implemented by CNNs that take a low-resolution image as input and generate a high-resolution version of the image as output. AI superresolution can be used to enhance the visual quality of an image and / or video and can be used for zoom and magnification. Petition 870260046470, dated 05 / 15 / 2026, p. 16 / 124 13 / 50 image scale. AI superresolution may be slower than bilinear and bicubic scaling, but overall produces much higher image quality.
[0029] Some applications that run on computing devices, and in particular on mobile computing devices (e.g., smartphones), such as map applications and some web browser applications, provide runtime zoom or runtime magnification functions for the content generated by the application. For example, some map applications enable zoom magnification of map content within the map application. Such applications support the ability to magnify content within the application in response to a user command to a touch-screen device or trackpad-type device (a user command to zoom). For example, a user command to zoom might include increasing the distance between two fingers in contact with a touch screen (sometimes called a pinch-back or pinch-out gesture), a double-tap and drag gesture, or another suitable user command.
[0030] However, computing devices typically do not provide runtime zoom functions outside of such applications, in part because neither bilinear / bicubic image scaling nor AI-based superresolution (CNN) are suitable for the task. Bilinear or bicubic image scaling typically does not provide sufficient content detail, and AI-based superresolution (CNN) does not provide smooth animations and consumes even more battery power at an undesirable rate. Petition 870260046470, dated 05 / 15 / 2026, page 17 / 124 14 / 50
[0031] For example, a CNN model can execute or generate each frame (e.g., in a graphics processing unit (GPU) or digital signal processor (DSP)) to provide hybrid scaling / zoom animations based on CNN superresolution. A typical hybrid CNN superresolution function causes significant power consumption and may have difficulty providing smooth animations (e.g., without visible frame drops), particularly for advanced display screens that provide display rates up to 120 Hz or 144 Hz.
[0032] Several embodiments include methods and computing devices configured to perform the methods of performing a zoom operation on a computing device. In some embodiments, a computing device processor can identify a region of interest (ROI) of an image presented on a computing device display based on a user command to zoom received by the computing device. In some embodiments, a computing device processor can use regional histogram analysis and / or image segmentation to identify the ROI. For example, a thumbnail image presented on a solid or low-texture background can be identified by the processor based on the pixel variability within the thumbnail image (as revealed in a histogram of the region) compared to the low pixel variability within the background.
[0033] In some embodiments, a processor in the computing device may determine a scaling factor for a final zoom-enlarged presentation. Petition 870260046470, dated 05 / 15 / 2026, page 18 / 124 15 / 50 of the ROI based on the information available to the processor. In some embodiments, the processor may determine the scaling factor based on how much the ROI can be scaled up and remain within the display dimensions or meet another image rendering requirement. In some embodiments, the processor may determine the scaling factor based on how quickly a final image can be generated using CNN processing methods and processors available on the computing device (called a CNN frame generation inference speed). In some embodiments, the computing device processor may determine (compute, identify) ROI dimensions (such as a width and height) and dimensions of a computing device display (such as a width and height, a number of vertical pixels and horizontal pixels, or other suitable dimensions).In some modes, the computing device can determine a time and / or a number of frames required to expand the... ROI for filling the available display dimensions, for example, in one or more dimensions.
[0034] In some embodiments, the computing device may identify dimensions of an image element within the ROI. For example, the computing device may identify a face within a photograph and may determine a time and / or number of frames required to expand the face (the image element within the ROI) to fill the available display dimensions. In some embodiments, the image rendering requirement may include a number of frames per second (FPS) generated by Petition 870260046470, dated 05 / 15 / 2026, page 19 / 124 16 / 50 scaling down of the final zoomed-in presentation for the generated sequence of intermediate-sized image frames. In some embodiments, the determination of the scaling factor for the final zoomed-in presentation of the ROI is additionally based on the dimensions of the ROI and the dimensions of the computing device display. In some embodiments, the determination of the scaling factor for the final zoomed-in presentation of the ROI may be additionally based on the dimensions of an image element within the ROI and the dimensions of the computing device display. In some embodiments, the computing device may determine the scaling factor for the final zoomed-in presentation in response to the detection of the start of a user zoom command (e.g., two-finger taps on a touchscreen display).
[0035] In some embodiments, the computing device processor may use CNN processes to generate the final zoomed-in presentation of the ROI for the given scaling factor. Then, to generate a zoom animation from the original ROI to the final zoomed-in presentation of the ROI, the computing device processor may downscale the final zoomed-in presentation to generate a sequence of intermediate-sized image frames, each having dimensions smaller than the dimensions of the final zoomed-in presentation of the ROI and larger than the dimensions of the ROI. In some embodiments, downscaling the final zoomed-in presentation to generate a sequence of intermediate-sized image frames may be accomplished using Petition 870260046470, dated 05 / 15 / 2026, page 20 / 124 17 / 50 a linear scaling technique. In some embodiments, the computing device can render an animation of changes in a displayed ROI size using the sequence of intermediate ROI image frames starting with the original ROI image and sequentially presenting increasingly larger intermediate image frames until the final zoomed-in presentation of the ROI is displayed. In some embodiments, the computing device can render the animation of changes in displayed ROI sizes using the sequence of intermediate image frames responsive to the user's zoom command.
[0036] How can a user focus attention on the ROI during a zoom animation? In some embodiments, the computing device processor renders the ROI zoom animation at a higher resolution than a zoom animation of surrounding images (e.g., background pixels). For example, in some embodiments, the processor may generate an enlarged-scale image frame sequence of the display outside the ROI using linear interpolation scaling techniques, while rendering the ROI zoom animation using the enlarged-scale CNN image and downscale frames, presenting on the display a combination of the intermediate ROI image frame sequence and the enlarged-scale image frame sequence of image elements outside the ROI.
[0037] For example, the computing device processor can identify an ROI using regional histogram analysis and image segmentation. The computing device processor can determine a factor of Petition 870260046470, dated 05 / 15 / 2026, page 21 / 124 18 / 50 scaling for a final zoomed-in presentation of the ROI (or for an image element within the ROI) by determining ROI dimensions (or image element dimensions), display dimensions, and / or a time required by the CNN to generate a zoomed-in presentation of the ROI or image element. For example, the computing device processor might determine that generating a zoomed-in presentation of the ROI or image element using a 2X scaling factor might require 27 milliseconds, generating a zoomed-in presentation of the ROI or image element using a 3X scaling factor might require 35 milliseconds, and generating a zoomed-in presentation of the ROI or image element using a 4X scaling factor might require 52 milliseconds.In some embodiments, the computing device processor may select a scaling factor based on one or more latency requirements, such as a frames per second threshold, a touch delay latency requirement, or another suitable latency requirement or threshold.
[0038] Continuing this example, the computing device processor can use CNN to generate a final zoomed-in presentation of the ROI or image element using the determined scaling factor. For example, if the computing device processor selects a scaling factor of 2X for an ROI with dimensions of 540 x 320, the computing device processor can zoom in or out of the ROI or image element using the 2X scaling factor to generate the final zoomed-in presentation of the ROI at 1080 x 640. The computing device processor can zoom out Petition 870260046470, dated 05 / 15 / 2026, page 22 / 12419 / 50 scales the final zoomed-in presentation of the ROI or image element to generate a sequence of intermediate-sized image frames. The intermediate-sized image frames can have dimensions that are larger than the ROI dimensions and smaller than the dimensions of the final zoomed-in presentation of the ROI. For example, the computing device processor might generate a first animation frame with dimensions of 550 x 324, a second animation frame with dimensions of 570 x 335, and so on, up to a final intermediate frame N. In some embodiments, the processor might use a linear scaling technique to generate the sequence of intermediate-sized image frames. The computing device processor can then render an animation of changes in the displayed size of the ROI or image element using the sequence of intermediate ROI image frames.In some modes, the computing device processor may stop the zoom animation before the final zoom image. In several modes, if the displayed content or the geometry of the content changes, the computing device processor may dynamically update the ROI and perform the various operations described above for the dynamically updated ROI.
[0039] In some embodiments, the computing device processor may use a combination of CNN and an interpolation scaling technique to generate a zoom animation for the ROI and a region outside the ROI. For example, the computing device processor may generate a sequence of enlarged image frames of display elements outside the ROI using a Petition 870260046470, dated 05 / 15 / 2026, page 23 / 124 20 / 50 interpolation scaling technique. The computing device's processor can then render an animation on the display that combines the sequence of intermediate image frames from the ROI and the sequence of upscaled image frames of image elements outside the ROI.
[0040] Several methods improve the user experience for users performing a zoom operation on an image element on a computing device, enabling a processor on the computing device to provide a runtime zoom function for almost any content presented by the computing device, and this is not limited to zoom operations performed within a specific application that depend on the design and functions of that specific application.
[0041] Figure 1 is a conceptual diagram that illustrates aspects of a method 100 for performing a zoom operation on a computing device according to various embodiments. The means for performing operations of method 100 may include one or more processors (e.g., processors 410, 412, 414, 416, 418, 421, 422, 452, 460 of Figure 4) in a computing device (e.g., 118, 400).
[0042] The computing device 118 can identify a region of interest (ROI) 122 of an image 120 presented on a display of the computing device 118 based on a user command to zoom received by the computing device 118 (for example, received by a touchscreen input device of the computing device 118). In some embodiments, the device of Petition 870260046470, dated 05 / 15 / 2026, page 24 / 124 21 / 50 computing 118 can use regional histogram analysis and / or image segmentation to identify ROI 122.
[0043] The computing device 118 can determine a scaling factor for a final zoomed-in presentation of the ROI 12 6 based on one or both of an image rendering requirement or a frame-generating convolutional neural network (CNN) inference speed (CNN model) 124. In some embodiments, the computing device 118 can determine (calculate, identify) dimensions of the ROI 122 (such as a width and height). In some embodiments, the computing device may determine the dimensions of a computing device display 118 (such as a width and height, a number of vertical and horizontal pixels, or other suitable dimensions). In some embodiments, the computing device may determine a time and / or number of frames required to expand the ROI 122 to fill the available display dimensions of the computing device 118.
[0044] In some embodiments, the computing device can identify dimensions of an image element within the ROI 122, such as a face or any other suitable image element. For example, the computing device 118 can identify a face within the ROI 122 and can determine a time and / or number of frames required to expand the face to fill the display dimensions of the computing device 118.
[0045] In some modes, the image rendering requirement may include a number of frames per second (FPS) generated by downscaling the final zoomed-in presentation to the Petition 870260046470, dated 05 / 15 / 2026, page 25 / 124 22 / 50 sequence generated from intermediate-sized image frames. In some embodiments, the determination of the scaling factor for the final zoom-enlarged presentation of the ROI 122 may additionally be based on the dimensions of the ROI 122 and the dimensions of the computing device display 118. In some embodiments, the determination of the scaling factor for the final zoom-enlarged presentation of the ROI 122 may additionally be based on the dimensions of an image element within the ROI 122 and the dimensions of the computing device display 118. In some embodiments, the computing device 118 may determine the scaling factor for the final zoom-enlarged presentation in response to the detection and initiation of a user zoom command.
[0046] Computing device 118 can use CNN 124 to generate the final zoomed-in presentation 126 of ROI 122 for the given scaling factor. Computing device 118 can use a linear scaler 128 to downscale the final zoom-enlarged presentation 126 to generate a sequence of intermediate-sized image frames 130, 132, 134, each having dimensions smaller than the dimensions of the final zoom-enlarged presentation 126 of ROI 122 and larger than the dimensions of ROI 122. The computing device can render an animation on a timeline of animation changes at a displayed size of ROI 122 using the sequence of intermediate image frames 130, 132, and 134. Based on aspects of the user's zoom command, such as command speed, command distance, or other magnitude, the animation can reach (ultimately display) the final zoom-enlarged presentation 126 of ROI Petition 870260046470, dated 05 / 15 / 2026, page 26 / 124 23 / 50 122, or the animation may not reach the final zoomed-in presentation 126 of ROI 122 and instead may reach (ultimately display) one of the intermediate image frames 130, 132, and 134.
[0047] Figures 2A and 2B illustrate an example of a neural network 200 suitable for implementation in a computing device according to various modalities. With reference to Figures 1 to 2A, the neural network 200 may include an input layer 202, intermediate layer(s) 204, and an output layer 206. Each of the layers 202, 204, 206 may include one or more processing nodes that receive input values, perform calculations based on the input values, and propagate the result (activation) to the next layer.
[0048] In feedforward neural networks, such as the 200 neural network, all calculations are performed as a sequence of operations on the outputs of a previous layer. The final set of operations generates the neural network's output, such as a probability that an image contains a specific item (e.g., dog, cat, etc.) or information indicating that a proposed action should be taken. The final output of the neural network may correspond to a task that the 200 neural network may be performing, such as determining whether an image contains a specific item (e.g., dog, cat, etc.). Many 200 neural networks are stateless. The output for an input is always the same, regardless of the sequence of inputs previously processed by the 200 neural network.
[0049] The 200 neural network can include fully connected (FC) layers, which are also sometimes called multi-layer perceptrons (MLPs). In a fully connected layer, Petition 870260046470, dated 05 / 15 / 2026, page 27 / 124 24 / 50: All outputs are connected to all inputs. The activation of each processing node is calculated as a weighted sum of all inputs received from the previous layer.
[0050] An example of computation performed by processing nodes and / or neural network 200 could be: yi=f(Í Equation (1) where Wij are weights, xi is the input to the layer, yj is the layer's output activation, f(·) is a nonlinear function and b is bias, which can vary with each node (e.g., bj). As another example, the 200 neural network can be configured to receive pixels from an image (i.e., input values) in the first layer and generate outputs indicating the presence of different low-level attributes (e.g., lines, edges, etc.) in the image. In a subsequent layer, these attributes can be combined to indicate the likely presence of higher-level attributes. For example, in training a neural network for scaling up an image, the output layer can generate a probability value that various lines, edges, colors, etc., are present in a scaled-up (i.e., increased-scale) version of the input image.In this way, the neural network can use the information available in a small, low-resolution image to predict which image elements should be added to fill in details between pixels as the image is enlarged.
[0051] The 200 neural network can be trained to scale up images using an image database. Petition 870260046470, dated 05 / 15 / 2026, page 28 / 124 25 / 50 of different sizes and resolutions. In this training process, a small image can be provided as the input, and a larger, higher-resolution image of the same subject can be provided as the expected / desired output. Learning is performed by comparing the output generated by the neural network 200 with the expected / desired output and by adjusting the weights. The difference between the expected / desired output and the output generated by the neural network 200 is called loss (L). The weights and biases in the neural network are then adjusted to approximate the output image to the expected output image provided, thus reducing the loss. During training, the weights (wij) can be updated using a scaling optimization process called gradient descent. This gradient indicates how the weights should change to reduce the loss (L). A multiple of the gradient of the loss with respect to each „ . . . . „ .1, , ,dL dL dL^ weight, which may be the partial derivative of the loss (e.g.g, ^p^£,^]^) in relation to weight, can be used to update weights and polarizations. This learning process can be repeated for a large image database.
[0052] Figure 2B illustrates a way to calculate partial derivatives of a gradient through a process called backpropagation. With reference to Figures 1 to 2B, backpropagation can operate by passing values backward through the network to calculate how the loss is affected by each weight. Backpropagation calculations can be similar to the calculations used when traversing the neural network in the forward direction (i.e., during inference). To improve performance, the loss (L) of multiple input datasets (a batch) can be collected and used in a single update pass. Petition 870260046470, dated 05 / 15 / 2026, page 29 / 124 26 / 50 of the weights. Many passes may be needed to train the neural network 200 with weights suitable for use during inference (e.g., at runtime or during the execution of a software application program).
[0053] The overall structure of neural network 200 and the operations of the processing nodes do not change as the neural network learns this task. After such training is completed, neural network 200 can process any image for upscaling using the determined weights and bias.
[0054] Figures 3A and 3B illustrate components of an example functionality that can be included in a 300 convolutional neural network, which can be implemented on a computing device that is configured to implement a generalized framework for continuous learning according to various modalities.
[0055] With reference to Figures 1 to 3B, the convolutional neural network 300 may include a first layer 301 and a second layer 311. Each layer 301, 311 may include one or more activation functions. In the example illustrated in Figure 3A, each layer 301, 311 includes a convolution functionality component 302, 312, a nonlinearity functionality component 304, 314, a normalization functionality component 306, 316, a clustering functionality component 308, 318, and a quantization functionality component 310, 320. It should be understood that, in various embodiments, the functionality components 302-310 or 312-320 may be implemented as part of a neural network layer or outside the neural network layer. It should also be understood that the illustrated order of operations in Petition 870260046470, dated 05 / 15 / 2026, page 30 / 124 27 / 50 Figure 3A is merely an example and is not intended to limit various modalities to any given order of operation. In various modalities, the order and / or inclusion of operations by functionality components 302-310 or 312-320 may change in any given layer. For example, normalization operations by normalization functionality component 306, 316 may come after convolution by convolution functionality component 302, 312 and before nonlinearity operations by nonlinearity functionality component 304, 314.
[0056] The convolutional functionality component 302, 312 can be an activation function for its respective layer 301, 311. The convolutional functionality component 302, 312 can be configured to generate an output activation matrix called an attribute map. The attribute maps generated in each successive layer 301, 311 typically include values that successively represent higher-level abstractions of input data (e.g., line, shape, object, etc.).
[0057] The nonlinearity functionality component 304, 314 can be configured to introduce nonlinearity in the output activation of its layer 301, 311. In various embodiments, this can be accomplished via a sigmoid function, a hyperbolic tangent function, a rectified linear unit (ReLU), a leaky ReLU, a parametric ReLU, an exponential LU (LU linear unit) function, a maxout function, swish, etc.
[0058] The normalization functionality component 306, 316 can be configured to control the distribution of input across layers to speed up the Petition 870260046470, dated 05 / 15 / 2026, page 31 / 124 28 / 50 training and improved accuracy of outputs or activations. For example, the distribution of inputs can be normalized to have a mean of zero and a standard deviation of one. The normalization function can also use batch normalization (BN) techniques to further scale and shift values to improve performance.
[0059] The clustering functionality components 308, 318 can be configured to decrease the dimensionality of an attribute map generated by the convolution functionality component 302, 312 and / or otherwise allow the convolutional neural network 300 to withstand small deviations and distortions in the values.
[0060] In some embodiments, the inputs to the first layer 301 may be structured as a set of three-dimensional input attribute maps 352 that form a channel of input attribute maps. For example, the convolutional neural network 300 may include a batch size of N three-dimensional attribute maps 352 with height H and width W, each having C number of input attribute map channels (illustrated as two-dimensional maps in C channels) and M three-dimensional filters 354 including C filters for each channel (also illustrated as two-dimensional filters for C channels). Applying the 1 to M filters 354 to the 1 to N three-dimensional attribute maps 352 results in N output attribute maps 356 that include M channels of width F and height E. As illustrated, each channel can be convolved with a three-dimensional filter 354.The results of these convolutions can be summed across all channels to generate the first layer 301 output activations in the form of an output attribute map channel 356. Filters. Petition 870260046470, dated 05 / 15 / 2026, page 32 / 124 29 / 50 additional three-dimensional features can be applied to the input attribute maps 352 to create additional output channels, and multiple input attribute maps 352 can be processed together as a batch to improve the reuse of filter weights. The output channel results (e.g., set of output attribute maps 356) can be provided to the second layer 311 in the convolutional neural network 300 for further processing. [00 61] Figure 4 is a component block diagram illustrating an example computing system implemented as a system in a package (SIP), 400 suitable for implementing various modalities. With reference to Figures 1 to 4, various modalities can be implemented in a variety of architectures that can be used in computing devices, including single-processor, multi-processor and multiprocessor computer systems, a system-on-chip (SOC), a SIP and any combination thereof.
[0062] The SIP 400 may include two SOCs 402, 404, a clock 406, and a voltage regulator 408. In some embodiments, the first SOC 402 may operate as the central processing unit (CPU) of the computing device, executing the instructions of software application programs by performing the arithmetic, logical, control, and input / output (I / O) operations specified by the instructions. In some embodiments, the second SOC 404 may operate as a specialized processing unit. For example, the second SOC 404 may operate as a specialized 5G processing unit responsible for managing high-volume, high-speed (e.g., 5) data. Petition 870260046470, dated 05 / 15 / 2026, page 33 / 124 30 / 50 Gbps, etc.) and / or very high frequency short wavelength (e.g., 28 GHz mmWave spectrum, etc.).
[0063] The first SOC 402 may include a digital signal processor (DSP) 410, a modem processor 412, a graphics processor 414, an application processor 416, one or more coprocessors 418 (e.g., vector coprocessor) connected to one or more of the processors, memory 420, a deep processing unit (DPU) 421, an AI processor 422, system components and resources 424, an interconnect / bus module 426, one or more temperature sensors 430, a thermal management unit 432, and a thermal power envelope (TPE) component 434. The second SOC 404 may include a 5G modem processor 452, a power management unit 454, an interconnect / bus module 464, and a plurality of mmWave transceivers. 456, 458 memory, and several additional 460 processors, such as an application processor, package processor, etc.
[0064] Each processor 410, 412, 414, 416, The 418, 421, 422, 452, and 460 models may include one or more cores, and each processor / core may perform operations independently of the other processors / cores. For example, the first 402 SOC may include a processor running a first type of operating system (e.g., FreeBSD, Linux, OS X, etc.) and a processor running a second type of operating system (e.g., Microsoft Windows 10). Furthermore, any or all of the 410, 412, 414, 416, 418, 421, 422, 452, and 460 processors may be included as part of a [missing information - likely a multiple-choice architecture]. Petition 870260046470, dated 05 / 15 / 2026, page 34 / 124 31 / 50 processor clustering (e.g., a synchronous processor clustering architecture, an asynchronous or heterogeneous processor clustering architecture, etc.).
[0065] The first and second 402, 404 SOCs may include various system components, features, and custom circuitry to manage sensor data, analog-to-digital conversions, wireless data transmissions, and to perform other specialized operations such as decoding data packets and processing encoded audio and video signals for rendering in a web browser. For example, the 424 system components and features of the first 402 SOC may include power amplifiers, voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, and other similar components used to support the processors and software clients running on a UE device.The components and resources of a 424 system may also include a set of circuits for interfacing with peripheral devices, such as cameras, electronic displays, wireless communication devices, external memory chips, etc.
[0066] The first and second SOCs 402, 404 can communicate via interconnect / bus module 450. The various processors 410, 412, 414, 416, 418 can be interconnected to one or more memory elements 420, system components and resources 424, and a thermal management unit 432 via an interconnect / bus module 426. Similarly, the processor Petition 870260046470, dated 05 / 15 / 2026, page 35 / 124 32 / 50 452 can be interconnected to the power management unit 454, the mmWave transceivers 456, the memory 458, and the various additional processors 460 via the interconnect / bus module 464. The interconnect / bus module 426, 450, 464 can include a reconfigurable logic gate array and / or implement a bus architecture (e.g., CoreConnect, AMBA, etc.). Communications can be provided by advanced interconnects, such as high-performance NoC networks.
[0067] The first and / or second SOCs 402, 404 may additionally include an input / output module (not shown) for communication with resources external to the SOC, such as a clock 406, a voltage regulator 408, a screen sensor unit 415, and a wireless transceiver 466 (e.g., wireless cellular transceiver, Bluetooth transceiver, etc.). The resources external to the SOC (e.g., clock 406, voltage regulator 408, screen sensor unit 415, wireless transceiver 466) may be shared by two or more of the internal SOC processors / cores.
[0068] In some embodiments, any or all of the processors 410, 412, 414, 416, 418, 421, 422, 452, and 460 can implement a CNN-based AI image processing system, a bilinear-bicubic image processing pipeline, and / or a convolutional neural network super-resolution (CNN SR) image processing pipeline. For example, in some embodiments, GPU 414 and / or DPU 421 can implement a bilinear-bicubic pipeline and / or AI processor 422, GPU 414 and / or DSP Petition 870260046470, dated 05 / 15 / 2026, page 36 / 124 33 / 50 410s can implement a CNN SR pipeline.
[0069] In some embodiments, any or all of the processors 410, 412, 414, 416, 418, 421, 422, 452, and 460 can be configured to work in conjunction with the screen sensor unit 415 to perform a zoom operation on a computing device display according to the modes. For example, the screen sensor unit 415 can monitor end-user touch gestures to conditionally trigger a dynamic continuous layer zoom and corresponding sequential scale-up animation frames at continuous variable scale-up step factors (1.0--1.01— 1.02--1.03-----xxxxx---------1.99, 2.00, 2.01-----xxxxx----3.0). The screen sensor unit 415 can detect a user command to zoom on an image layer within the display presented on a computing device electronic display and send the information to the GPU 414 and the AI processor 422.The GPU 414 can receive a user command to zoom and use an interpolation scaling technique in a first image processing pipeline to adjust the displayed size of the image layer. In parallel, the AI processor 422 can use a superresolution technique in a second image processing pipeline to adjust the displayed size of the image layer. The AI processor 422, the GPU 414, and / or another processor (e.g., application processor 416) can upscale an output from the interpolation scaling technique in the first image processing pipeline with an output from the superresolution technique in the second processing pipeline. Petition 870260046470, dated 05 / 15 / 2026, page 37 / 124 34 / 50 images to generate a frame with an upscaled image layer and have an electronic display of a computing device show the upscaled frame of the image layer.
[0070] Figure 5A illustrates a method 500a for performing a zoom operation on a computing device according to various embodiments. With reference to Figures 1 to 5A, the means for performing operations of method 500a may include one or more processors (e.g., processors 410, 412, 414, 416, 418, 421, 422, 452, 460) in a computing device (e.g., 118, 400).
[0071] In block 502, the processor can identify an ROI from an image presented on a computing device display based on a user zoom command received by the computing device. In some embodiments, the processor can use regional histogram analysis and / or image segmentation to identify the ROI. In some cases, the processor can identify a particular element (e.g., a face) within an identified ROI, such as using image processing and recognition techniques.
[0072] In block 504, the processor can determine a scaling factor for a final zoom-enlarged ROI display based on one or both of an image rendering requirement or a frame generation CNN inference speed. In some embodiments, the processor can determine the scaling factor for the final zoom-enlarged display in response to the detection of the start of a user zoom command. In some embodiments, the processor can Petition 870260046470, dated 05 / 15 / 2026, page 38 / 124 35 / 50 Use ROI dimensions and computing device display dimensions in determining the scaling factor for the final zoom-in presentation of the ROI. In some embodiments, the processor may use dimensions of an image element within the ROI and computing device display dimensions in determining the scaling factor for the final zoom-in presentation of the ROI. In some embodiments, the image rendering requirement may include a number of frames per second (FPS) generated by downscaling the final zoom-in presentation to the generated sequence of intermediate-sized image frames.
[0073] In block 506, the processor can use CNN to generate the final zoomed-in presentation of the ROI for the given scaling factor. In this way, a final frame of a zoom animation at a given zoom factor or maximum zoom factor can be generated before the animation's image frames are generated. Thus, instead of incrementally increasing the size of the ROI in each image frame of the animation sequence, the processor can first generate the final image using CNN to render the image in fine detail. Although generating this final zoomed-in presentation of the ROI can take several milliseconds using CNN techniques, this processing can be performed within the time between when a user first touches a touchscreen display and when the user spreads their fingertips in a user command to zoom to a perceptible degree.In other words, in the time it takes for a user to start moving their fingertips in response to a user command to zoom, the processor may have generated a... Petition 870260046470, dated 05 / 15 / 2026, page 39 / 124 36 / 50 final ROI presentation via Zoom.
[0074] In block 508, the processor can downscale the final zoomed-in presentation of the ROI to generate a sequence of intermediate-sized image frames, each having dimensions smaller than the dimensions of the final zoomed-in presentation of the ROI and incrementally larger than the dimensions of the ROI. In some embodiments, the processor can use a linear scaling technique to perform the downscaling of the final zoomed-in presentation to generate the sequence of intermediate-sized image frames. Given that such downscaling starts from a high-resolution final ROI image generated using CNN processes, each of the downscaled image frames will exhibit high resolution.
[0075] In block 510, the processor can render an animation of changes in the displayed size of the ROI using the sequence of intermediate image frames of the ROI. In particular, downscaled image frames can be rendered starting with the smallest image frame with subsequently rendered image frames of increasing size until the final zoomed-in presentation of the ROI is displayed. In some embodiments, the processor can render the animation of changes in the displayed size of the ROI using the sequence of intermediate image frames in response to the user's zoom command. In other words, the processor can match the rate at which increasing (or decreasing) size image frames are presented on the display to the rate at which the user moves their fingertips apart (to zoom out) or Petition 870260046470, dated 05 / 15 / 2026, page 40 / 124 37 / 50 together (to increase zoom). In this way, the sequence of intermediate image frames generated in block 508 can be displayed in order and at a rate that corresponds (i.e., is responsive) to the user's zoom command.
[0076] Figure 5B illustrates 500b operations that can be performed as part of method 500a to perform a zoom operation on a computing device according to various modes. 500b operations can be useful when the ROI is present on a display that includes other attributes and / or a background that is not of interest for zoom magnification. With reference to Figures 1 to 5B, the means to perform 500b operations may include one or more processors (e.g., processors 410, 412, 414, 416, 418, 421, 422, 452, 460) on a computing device (e.g., 118, 400).
[0077] After downscaling the final zoomed-in display to generate the sequence of intermediate-sized image frames, each having dimensions smaller than the dimensions of the final zoomed-in display of the ROI and larger than the dimensions of the ROI in block 508 of method 500a as described, the processor can generate an upscaled image frame sequence of display elements outside the ROI using an interpolation (or linear) scaling technique in block 520.
[0078] In block 522, the processor can then render an animation on the display that combines the sequence of intermediate image frames from the ROI and the sequence of upscaled image frames of image elements outside the ROI.
[0079] Figure 6 is a block diagram of Petition 870260046470, dated 05 / 15 / 2026, page 41 / 124 38 / 50 components of a computing device suitable for use with various embodiments. With reference to Figures 1 to 6, in some embodiments, the computing device may be implemented in the form of a laptop-type computer 600. In several embodiments, the laptop-type computer 600 may include a touchpad (or trackpad) type touch surface 617 that serves as the computer's pointing device and, in this way, may receive pinch-in, pinch-out, dragging, scrolling, tapping, etc. gestures similar to those that may be implemented in computing devices equipped with a touch-screen display. The laptop-type computer 600 may include a processor 602 coupled to volatile memory 612 and high-capacity non-volatile memory, such as a Flash memory disk drive 613.Additionally, the laptop-type computer 600 may include one or more antennas 608 for sending and receiving electromagnetic radiation that may be connected to a wireless data link and / or mobile phone transceiver 616 coupled to the processor 602. The laptop-type computer 600 may also include a transceiver 614 that implements short-range wireless communication using a variety of short-range communication protocols, such as any of the Institute of Electrical and Electronics Engineers (IEEE) 802.11 and 802.15 protocols. The laptop-type computer 600 may also include a compact disc drive (CD) 615 coupled to the processor 602. The laptop-type computer 600 may include a touchpad 617, a keyboard 618, and a display 619, all of them. Petition 870260046470, dated 05 / 15 / 2026, page 42 / 124 39 / 50 coupled to the 602 processor. Other configurations of the 600-type laptop computer may include a computer mouse or trackball coupled to the processor (for example, via a Universal Serial Bus (USB) input), as are well known, which can also be used in conjunction with various modes.
[0080] Figure 7 is a component block diagram of a computing device suitable for use in various embodiments. With reference to Figures 1 to 7, in some embodiments, the computing device may be implemented in the form of a smartphone 700. The smartphone 700 may include a first set of circuits 402 coupled to a second set of circuits 404. The first and second SoCs 402, 404 may be coupled to internal memory 716, a display 712, and a speaker 714. The display 712 may include or incorporate a touch-sensitive input device configured to receive input such as pinch-in, pinch-out, drag, scroll, tap gestures, etc.The first and second sets of 402, 404 circuits can also be coupled to at least one subscriber identity module (SIM) 740 and / or a SIM interface that can store information supporting a first 5GNR subscription and a second 5GNR subscription, which support a service on a non-standalone 5G network (NSA).
[0081] The 700 smartphone may include a 704 antenna for sending and receiving electromagnetic radiation that may be connected to a 466 wireless transceiver coupled to one or more processors in the first and / or second circuit sets 402, 404. The 700 smartphone may also include Petition 870260046470, dated 05 / 15 / 2026, p. 43 / 124 40 / 50 menu selection buttons or rocker switches 720 to receive user input. The smartphone 700 may also include a sound encoding / decoding circuit 710, which digitizes sound received from a microphone into data packets suitable for wireless transmission and decodes received sound data packets to generate analog signals that are provided to the speaker to generate sound. In addition, one or more of the processors in the first and second circuit sets 402, 404, the wireless transceiver 466 and the CODEC 710 may include a digital signal processor (DSP) circuit 740.
[0082] Figure 8 is a component block diagram of a computing device suitable for use with various embodiments. With reference to Figures 1 to 8, in some embodiments, the computing device can be implemented in the form of an 800 smart watch.
[0083] The 800 smart clock may include an 802 SoC which includes two or more processors (e.g., application processor, low-power processor) coupled to 804 and 806 internal memories. The 804 and 806 internal memories may be volatile or non-volatile memories, and may also be secure and / or encrypted memories, or insecure and / or unencrypted memories, or any combination thereof. The 802 SoC may also be coupled to an 820 touch-screen display, such as a resistive touch-screen, a capacitive touch-screen, an infrared touch-screen, or similar. Additionally, the 800 smart clock may have one or more 808 antennas for sending and receiving Petition 870260046470, dated 05 / 15 / 2026, p. 44 / 124 41 / 50 electromagnetic radiation that can be connected to one or more wireless data links 812, such as one or more Bluetooth® transceivers that can be coupled to the SoC 802.
[0084] The smartwatch 800 may also include physical and / or virtual buttons 822 and 810 for receiving user input, as well as a slide sensor 816 for receiving user input. The touchscreen display 820 may be coupled to a touchscreen display interface module that is configured to receive signals from the touchscreen display 820 indicating the locations on the screen where the user's fingertip or a stylus is touching the surface and send information about the coordinates of the touch events to the SoC 802. The physical and / or virtual buttons 822 and the touchscreen display 820 may be configured to receive input such as pinch-in, pinch-out, drag, scroll, tap gestures, etc.In addition, the 802 SoC can be configured with processor-executable instructions to correlate images displayed on the 820 touchscreen display with the location of touch events received from the touchscreen interface module to detect when a user has interacted with a graphical interface icon, such as a virtual button.
[0085] The 802 SoC can be any programmable microprocessor, microcomputer, or multiprocessor chip or chips that can be configured by software instructions (applications) to perform a variety of functions, including multimodal functions. In some devices, multiple processors may be provided, such as one processor performing functions of Petition 870260046470, dated 05 / 15 / 2026, page 45 / 124 42 / 50 wireless communication and a dedicated processor running other applications. Typically, software applications can be stored in internal memory before being accessed and loaded into the 802 SoC. The 802 SoC may include sufficient internal memory to store the application's software instructions. In many devices, the internal memory may be volatile or non-volatile memory, such as flash memory, or a mixture of both. For the purposes of this description, a general reference to memory refers to the memory accessible by the 802 SoC, including internal memory or removable memory connected to the wearable device and the memory within the 802 SoC itself.
[0086] The processors of laptop-type computers 600, smartphones 700, and smartwatches 800 can be any programmable microprocessor, microcomputer, or chip or multiprocessor chips that can be configured by software instructions (applications) to perform a variety of functions, including the functions of various modes described. In some computing devices, multiple processors may be provided, such as one processor within a first set of circuits dedicated to wireless communication functions and one processor within a second set of circuits dedicated to executing other applications. Software applications may be stored in memory before being accessed and loaded into the processor. Processors may include sufficient internal memory to store the application software instructions.
[0087] Implementation examples are described in the following paragraphs. Although some of the examples of Petition 870260046470, dated 05 / 15 / 2026, page 46 / 124 43 / 50 implementations described below in terms of example methods, other example implementations may include: the example methods discussed in the following paragraphs implemented by a computing device that includes a processor configured with processor-executable instructions to perform operations of the methods in the following example implementations; the example methods discussed in the following paragraphs implemented by a computing device including means for performing functions of the methods in the following example implementations; and the example methods discussed in the following paragraphs may be implemented as a processor-readable non-transient storage medium that stores processor-executable instructions configured to cause a processor in a computing device to perform the operations of the methods in the following example implementations.
[0088] Example 1. A method for performing a zoom operation on a computing device, including identifying a region of interest (ROI) of an image presented on a computing device display based on a user command to zoom received by the computing device, determining a scaling factor for a final zoomed-in presentation of the ROI based on one or both of an image rendering requirement or a frame-generating convolutional neural network (CNN) inference speed, using the CNN to generate the final zoomed-in presentation of the ROI for the determined scaling factor, downscaling the final zoomed-in presentation to generate a sequence of intermediate-sized image frames, each having Petition 870260046470, dated 05 / 15 / 2026, page 47 / 124 44 / 50 dimensions smaller than the final ROI zoom presentation dimensions and larger than the ROI dimensions, and render an animation of changes at a displayed ROI size using the ROI intermediate image frame sequence.
[0089] Example 2. The method from example 1, where the image rendering requirement includes a number of frames per second (FPS) generated by downscaling the final zoomed-in presentation to the generated sequence of intermediate-sized image frames.
[0090] Example 3. The method of either example 1 or 2, in which the downscaling of the final zoomed-in presentation to generate a sequence of intermediate-sized image frames is performed using a linear scaling technique.
[0091] Example 4. The method of any of examples 1 to 3, wherein the determination of the scaling factor for the final zoom-enlarged presentation and the use of the CNN to generate the final zoom-enlarged presentation are performed in response to the detection of a user zoom command start, and the rendering of the animation of changes in the displayed size of the ROI using the intermediate image frame sequence is performed in response to the user zoom command.
[0092] Example 5. The method of any of examples 1 to 4, wherein the determination of the scaling factor for the final zoomed-in presentation of the ROI is additionally based on the dimensions of the ROI and the dimensions of the computing device display.
[0093] Example 6. The method of any of the Petition 870260046470, dated 05 / 15 / 2026, page 48 / 124 45 / 50 examples 1 to 5, where the determination of the scaling factor for the final zoomed-in presentation of the ROI is additionally based on the dimensions of an image element within the ROI and the dimensions of the computing device display.
[0094] Example 7. The method of any of examples 1 to 6, including additionally generating a sequence of upscaled image frames of display elements outside the ROI using an interpolation scaling technique, wherein rendering the animation of changes in the displayed size of the ROI using the intermediate image frame sequence of the ROI includes rendering an animation on the display that combines the intermediate image frame sequence of the ROI and the upscaled image frame sequence of image elements outside the ROI.
[0095] As used in this application, the terms component, module, system, and the like are intended to include a computer-related entity such as, but not limited to, hardware, firmware, a combination of hardware and software, software, or running software, which are configured to perform particular operations or functions. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, an execution thread, a program, and / or a computer. By way of illustration, both an application running on a computing device and the computing device itself may be referred to as a component. One or more components may reside in a process and / or execution thread, and a component may be located on a processor or core and / or distributed among Petition 870260046470, dated 05 / 15 / 2026, page 49 / 124 46 / 50 two or more processors or cores. Furthermore, these components can be executed from various non-transient computer-readable media that have various instructions and / or data structures stored in them. The components can communicate through local and / or remote processes, function or procedure calls, electronic signals, data packets, memory read / write, and other known network, computer, processor, and / or process-related communication methodologies.
[0096] Several embodiments illustrated and described are provided merely as examples to illustrate various attributes of claims. However, the attributes shown and described in relation to any given embodiment are not necessarily limited to the associated embodiment and may be used or combined with other embodiments that are shown and described. Additionally, the claims are not intended to be limited by any example embodiment. For example, one or more of the method operations may be replaced or combined with one or more method operations.
[0097] The above method descriptions and process flow diagrams are provided merely as illustrative examples and are not intended to require or imply that the operations of various embodiments must be performed in the order presented. As will be recognized by one skilled in the art, the order of operations in the preceding embodiments may be performed in any order. Words such as after this, then, afterward, etc., are not intended to limit the order of operations; these words are used simply to guide the reader through the description of the methods. Additionally, Petition 870260046470, dated 05 / 15 / 2026, p. 50 / 124 47 / 50 Any reference to claim elements in the singular, for example, using the articles a, an or the, should not be interpreted as limiting the element to the singular.
[0098] The various illustrative logic blocks, modules, circuits, and algorithmic operations described in connection with the embodiments disclosed in the present invention can be implemented in the form of electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and operations have been described above in terms of their functionality. The possibility of such functionality being implemented as hardware or software depends on the particular application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the claims.
[0099] The hardware used to implement the various logics, logic blocks, modules and illustrative circuits described in connection with the embodiments disclosed in the present invention may be implemented or realized with a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components or any Petition 870260046470, dated 05 / 15 / 2026, page 51 / 124 48 / 50 combination of these designed to perform the functions described in the present invention. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other similar configuration. Alternatively, some operations or methods may be performed by a set of circuits that is specific to a given function.
[00100] In one or more embodiments, the described functions can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored as one or more instructions or code in a computer-readable non-transient medium or a processor-readable non-transient medium. The operations of a method or algorithm disclosed in the present invention can be incorporated into a processor-executable software module, which may reside in a computer-readable non-transient storage medium. Computer-readable non-transient storage media can be any storage media that can be accessed by a computer or a processor. By way of example, but not limitation, such computer-readable non-transient storage media may include RAM, ROM, EEPROM, Petition 870260046470, dated 05 / 15 / 2026, p. 52 / 124 49 / 50 Flash memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other means that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. Disks (disk and disc), as used in the present invention, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically by means of lasers. Combinations of the foregoing are also included within the scope of computer-readable and processor-readable non-transient media.Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and / or instructions in a non-transient, processor-readable medium and / or a computer-readable medium, which may be incorporated into a computer program product.
[00101] The foregoing description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles set forth in the present invention may be applied to other embodiments without departing from the scope of the claims. Thus, the present disclosure is not intended to be limited to the embodiments shown in the present invention, but should be consistent with the broader scope consistent with Petition 870260046470, dated 05 / 15 / 2026, page 53 / 124 50 / 50 the following claims and the innovative principles and attributes disclosed in the present invention. Petition 870260046470, dated 05 / 15 / 2026, p. 54 / 124
Claims
1 / 5 CLAIMS 1. A method for performing a zoom operation on a computing device, characterized in that it comprises: identifying a region of interest, ROI, of an image presented on a display of the computing device based on a user command to zoom received by the computing device; determining a scaling factor for a final zoomed-in presentation of the ROI based on one or both of an image rendering requirement or an inference speed of a frame-generating convolutional neural network, CNN; using the CNN to generate the final zoomed-in presentation of the ROI for the determined scaling factor; downscaling the final zoomed-in presentation to generate a sequence of intermediate-sized image frames, each having dimensions smaller than the dimensions of the final zoomed-in presentation of the ROI and larger than the dimensions of the ROI;and render an animation of changes at a displayed ROI size using the ROI's intermediate image frame sequence.
2. A method according to claim 1, characterized in that the image rendering requirement comprises a number of frames per second (FPS) generated by downscaling the final zoomed-in presentation to the generated sequence of intermediate-sized image frames.
3. Method according to claim 1, Petition 870260046470, dated 05 / 15 / 2026, page 55 / 124 2 / 5 characterized in that the downscaling of the final zoomed-in presentation to generate the sequence of intermediate-sized image frames is performed using a linear scaling technique.
4. A method according to claim 1, characterized in that: the determination of the scaling factor for the final zoom-enlarged presentation and the use of CNN to generate the final zoom-enlarged presentation are performed in response to the detection of a user zoom command initiation; and the rendering of the animation of changes in the displayed size of the ROI using the sequence of intermediate image frames is performed in response to the user zoom command.
5. Method, according to claim 1, characterized in that the determination of the scaling factor for the final zoomed-in presentation of the ROI is additionally based on the dimensions of the ROI and the dimensions of the computing device display.
6. Method, according to claim 1, characterized in that the determination of the scaling factor for the final zoomed-in presentation of the ROI is further based on the dimensions of an image element within the ROI and the dimensions of the computing device display.
7. Method according to claim 1, characterized in that it further comprises: generating an enlarged-scale image frame sequence of display elements outside the ROI using an interpellation scaling technique, wherein rendering the animation of changes in the displayed size of the ROI using the intermediate image frame sequence of the ROI comprises rendering an animation on the display that combines the intermediate image frame sequence of the ROI and the enlarged image frame sequence of image elements outside the ROI.
8. Computing device characterized in that it comprises: means for identifying a region of interest (ROI) of an image presented on a display of the computing device based on a user command to zoom received by the computing device; means for determining a scaling factor for a final zoomed-in presentation of the ROI based on one or both of an image rendering requirement or an inference speed of a frame-generating convolutional neural network (CNN); means for using the CNN to generate the final zoomed-in presentation of the ROI for the determined scaling factor; means for downscaling the final zoomed-in presentation to generate a sequence of intermediate-sized image frames, each having dimensions smaller than the dimensions of the final zoomed-in presentation of the ROI and larger than the dimensions of the ROI;and means for rendering an animation of changes at a displayed ROI size using the ROI's intermediate image frame sequence.
9. Computing device, according to Petition 870260046470, dated 05 / 15 / 2026, page 57 / 124 4 / 5 claim 8, characterized in that the image rendering requirement comprises a number of frames per second, FPS, generated by downscaling the final zoomed-in presentation to the generated sequence of intermediate-sized image frames.
10. Computing device, according to claim 8, characterized in that the means for downscaling the final zoomed-in display to generate the intermediate-sized image frame sequence uses a linear scaling technique.
11. Computing device according to claim 8, characterized in that: the means for determining the scaling factor for the final zoom-enlarged presentation and using the CNN to generate the final zoom-enlarged presentation comprise means for determining the scaling factor for the final zoom-enlarged presentation and using the CNN to generate the final zoom-enlarged presentation in response to the detection of a user zoom command initiation; and the means for rendering the animation of changes in the displayed size of the ROI using the intermediate image frame sequence comprise means for rendering the animation of changes in the displayed size of the ROI using the intermediate image frame sequence responsive to the user zoom command.
12. Computing device, according to claim 8, characterized in that the means for determining the scaling factor for the final zoom enlarged presentation of the ROI comprise means for determining the scaling factor for the final zoom enlarged presentation of the ROI based on the dimensions of the ROI and the dimensions of the computing device display.
13. Computing device, according to claim 8, characterized in that the means for determining the scaling factor for the final zoom-enlarged presentation of the ROI comprise means for determining the scaling factor for the final zoom-enlarged presentation of the ROI based on the dimensions of an image element within the ROI and the display dimensions of the computing device.
14. Computing device according to claim 8, characterized in that it further comprises: means for generating an enlarged-scale image frame sequence of display elements outside the ROI using an interpolation scaling technique, wherein the means for rendering the animation of changes in the displayed size of the ROI using the intermediate image frame sequence of the ROI comprise means for rendering an animation on the display that combines the intermediate image frame sequence of the ROI and the enlarged image frame sequence of image elements outside the ROI.
15. Computer-readable memory characterized in that it comprises instructions stored therein, the instructions being executable by the computer to perform the method as defined in any one of claims 1 to 7. Petition 870260046470, dated 05 / 15 / 2026, p. 59 / 124
