Display method, device, chip system and storage medium of grid interface

Abstract

Embodiments of the present application provide a kind of palace grid interface display method, equipment, chip system and storage medium, involve terminal technical field.In the present application scheme, based on the AP value of the picture in the first palace grid in current palace grid interface and the AP value of the picture in the last palace grid, the AP range of the picture possibly used when drawing target palace grid interface (including N column palace grid) is predicted, and the picture in AP range is spliced, obtain a spliced picture, 1 line includes 2N-1 picture, the AP value of the picture of adjacent palace grid in 1 line is 1, the AP value of the picture of the jth row i palace grid and the AP value of the picture of the j+1th row i palace grid are different N.In switch from current gear to target gear, based on scaling center, a big picture is cut out from spliced picture, and this big picture is drawn into View control.The number of drawing picture is reduced to 1 from multiple, the number of drawing picture is reduced to 1 time, and the drawing time is shortened, and the processing speed is improved.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a display method, device, chip system and storage medium for a grid interface. Background Technology

[0002] With the rapid development of terminal technology, the storage space of mobile phones and other electronic devices is becoming increasingly larger. Gallery applications on electronic devices can store a large number of images. Gallery applications can set up multiple albums based on image type, such as camera albums, all photos albums, all videos albums, screenshot albums, and personal favorites albums.

[0003] Currently, electronic devices support displaying a fixed number of columns of images in a grid format within a photo album page. In related rendering schemes, electronic devices typically render the images in each grid cell sequentially. However, when an album page contains a large number of images, the central processing unit (CPU) issues numerous rendering instructions to the graphics processing unit (GPU). This requires the GPU to render a large number of images, resulting in prolonged processing time and causing stuttering or frame drops in the grid interface, negatively impacting the user experience. Summary of the Invention

[0004] This application provides a method, device, chip system, and storage medium for displaying a grid interface, which solves the technical problem that related solutions are time-consuming when a grid interface includes a large number of images.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] Firstly, embodiments of this application provide a method for displaying a grid interface. The method includes:

[0007] Receive the user's first operation on the electronic device;

[0008] In response to the first operation, the first grid interface is displayed, and the first stitched image is obtained based on the adaptation position values ​​of the images in the first grid interface. The first grid interface may include M columns of grids, each row of the first stitched image may include 2N-1 images, the adaptation position values ​​of adjacent grids in each row of the first stitched image differ by 1, and the adaptation position value of the image in the j-th row and i-th grid of the first stitched image differs from the adaptation position value of the image in the (j+1)-th row and i-th grid by N, where i and j are positive integers, and N is not equal to M.

[0009] Receive a second operation from the user on the electronic device;

[0010] In response to the second operation, a first image is cropped from the first stitched image, and a second grid interface corresponding to the first image is displayed. Each row of the first image may include N images, and the second grid interface may include N columns.

[0011] In the above scheme, the first grid interface can include P rows and M columns, the second grid interface can include Q rows and N columns, each column of the first image can include Q images, and each row of the first image can include N images. P, M, Q, and N are all positive integers, N is not equal to M, and P is not equal to Q.

[0012] If a traditional grid-based interface rendering scheme is used, and there are N*Q images to be displayed in a single grid frame, then the GPU will render N*Q images and perform N*Q rendering operations. However, the scheme provided in this application predicts the AP range of images that may be used when rendering the second grid, and stitches the images within the AP range together to obtain a stitched image. By cropping a larger image from the stitched image corresponding to the N*Q images to be displayed, the number of images rendered by the GPU can be reduced from N*Q to 1, and the number of rendering operations can be reduced to 1, thereby shortening the grid-based interface rendering time and improving the GPU's processing speed. It is understandable that the larger the value of N*Q, the more significant the speed improvement in the scheme provided in this application compared to the traditional grid-based interface rendering scheme.

[0013] In one possible implementation, this application embodiment involves six parameters: curSpanCount, targetSpanCount, curRowCount, targetRowCount, canvasSpanCount, and canvasRowCount. Here, curSpanCount represents the number of columns in the grid interface for the current gear position, targetSpanCount represents the number of columns in the grid interface for the target gear position, curRowCount represents the number of rows in the grid interface for the current gear position, targetRowCount represents the number of rows in the grid interface for the target gear position, canvasSpanCount represents the number of columns in the stitched image, and canvasRowCount represents the number of rows in the stitched image.

[0014] (a) When switching from a smaller gear with fewer grid columns to a larger gear with more grid columns, the following relationship must be satisfied:

[0015] canvasSpanCount=targetSpanCount*2-1;

[0016] canvasRowCount=targetRowCount*2-(curSpanCount*curRowCount / targetSpanCount).

[0017] Where targetSpanCount is N, targetRowCount is Q, curSpanCount is M, and curRowCount is P.

[0018] When switching from M mode to N mode (N > M), the second grid interface needs to satisfy the following: the AP values ​​of adjacent grids in a row differ by 1, and the AP value of the image in the j-th row, ith grid differs from the AP value of the image in the (j+1)-th row, ith grid by N. When the zoom center is any grid in the first grid interface, in order to crop the large image corresponding to the grid where the zoom center is located from the first stitched image, it needs to satisfy the following: when any AP value in N mode is used as the AP value of the grid in the first row, first column of the large image, there are N-1 grids to the right of this AP value. Based on this, each column in the first stitched image includes 2N-1 images.

[0019] When the zoom center is any cell in the small-grid interface, in order to crop the large image corresponding to the cell where the zoom center is located from the stitched image, the following conditions must be met: the stitched image must include at least the large image with the first row and first column of the current grid interface as the zoom center, and the large image with the last row and last column of the current grid interface as the zoom center. It can be understood that when switching from a small-grid interface with fewer columns to a large-grid interface with more columns, the number of cells in the grid interface increases. The first row and first column of the current grid interface involves the maximum value of the AP range in the stitched image, and the last row and last column of the current grid interface involves the minimum value of the AP range in the stitched image. Therefore, when the stitched image includes the large image with the first row and first column of the current grid interface as the zoom center, and the large image with the last row and last column of the current grid interface as the zoom center, if the zoom center is located in any cell in the current grid interface, then the electronic device can crop the corresponding large image from the stitched image. Based on this, each column in the first stitched image contains Q*2-(M*P / N) images.

[0020] (ii) When switching from a gear with more grid columns to a gear with fewer grid columns, the following relationship must be satisfied:

[0021] canvasSpanCount=targetSpanCount*2-1;

[0022] canvasRowCount=curSpanCount*curRowCount / targetSpanCount.

[0023] Where targetSpanCount is N, curSpanCount is M, and curRowCount is P.

[0024] When switching from M to N (N < M), the second grid interface needs to satisfy the following: the AP values ​​of adjacent grids in a row differ by 1, and the AP value of the image in the j-th row, ith grid differs from the AP value of the image in the (j+1)-th row, ith grid by N. When the zoom center is any grid in the larger zoom level's grid interface, in order to crop the larger image corresponding to the grid where the zoom center is located from the stitched image, it needs to satisfy the following: when any AP value of the target zoom level is used as the AP value of the grid in the first row, first column of the larger image, there are N-1 grids to the right of this AP value. Based on this, each column in the first stitched image can include 2N-1 images.

[0025] When the zoom center is any cell in the large-scale grid interface, in order to crop the large image corresponding to the cell where the zoom center is located from the stitched image, the following conditions must be met: the stitched image must include at least the large image with the first row and first column of the current grid interface as the zoom center, and the large image with the last row and last column of the current grid interface as the zoom center. It can be understood that when switching from a large to a small scale, the number of cells in the grid interface decreases. The first row and first column of the current grid interface involves the minimum value of the AP range in the stitched image, and the last row and last column of the current grid interface involves the maximum value of the AP range in the stitched image. Therefore, when the stitched image includes the large image with the first row and first column of the current grid interface as the zoom center, and the large image with the last row and last column of the current grid interface as the zoom center, if the zoom center is located in any cell in the current grid interface, then the electronic device can crop the corresponding large image from the stitched image. Based on this, each column in the first stitched image contains M*P / N images.

[0026] In one possible implementation, the adaptation position value of the center cell of the first stitched image is equal to the adaptation position value of the center cell of the first grid interface. The adaptation position value of the center cell of the first grid interface is determined based on the adaptation position values ​​of the first cell and the last cell of the first grid interface. Here, the center cell refers to the cell located in the middle row and middle column of the day view, month view, or year view.

[0027] For example, if the AP of the first cell is denoted as startAP and the AP of the last cell is denoted as endAP, then the AP value of the center cell is represented as: (startAP + endAP1) / 2. For instance, if startAP = 100 and endAP = 123, then the AP value of the center cell is: or,

[0028] In one possible implementation, the first operation and the second operation can be any of the following:

[0029] ①The first operation is to click on the album card, and the second operation is to increase the distance between two fingers;

[0030] ②The first operation is a swipe operation on the third grid interface, and the second operation is an operation to increase or decrease the distance between two fingers. The third grid interface can include M columns of grids. The adaptation position value of the image in the third grid interface is not exactly the same as the adaptation position value of the image in the first grid interface.

[0031] ③The first operation is to increase the distance between two fingers, and the second operation is to decrease the distance between two fingers;

[0032] ④ The first operation is to reduce the distance between two fingers, and the second operation is to increase the distance between two fingers.

[0033] In the above solution, when a user clicks on an album card, swipes on a grid screen, or changes the zoom trend (from decreasing to increasing the two-finger spacing, or vice versa), these actions may trigger the electronic device to update the AP value of the grid screen. After updating the AP value of the grid screen, the stitched image is reloaded based on the current AP value, so that when a zoom operation is received subsequently, a suitable large image can be cropped from the stitched image.

[0034] In one possible implementation, the second grid interface is drawn based on a first view control, the size of which is greater than or equal to the screen size. Accordingly, cropping the first image from the first stitched image may include: determining the drawing area of ​​the second grid interface within the first view control; determining the adaptation position values ​​of the first and last grid cells in the drawing area based on the layout and adaptation position values ​​of the selected grid cells at the zoom center, with each adaptation position value corresponding to one image; cropping the first image from the first stitched image based on the adaptation position values ​​of the first and last grid cells in the drawing area, where the adaptation position value of the first grid cell in the first image is equal to the adaptation position value of the first grid cell in the drawing area, and the adaptation position value of the last grid cell in the first image is equal to the adaptation position value of the last grid cell in the drawing area; and drawing the first image onto the drawing area to obtain the second grid interface.

[0035] In the above scheme, for zoom operation scenarios, when a user's zoom operation is received, the layout position value and adaptation position value of the selected grid at the zoom center can be used to determine which row and column of the drawing area of ​​the next level in the first view control, and keep the AP value of the zoom center unchanged, while refreshing the AP values ​​of each grid before and after the zoom center, so that the order of the images displayed on the screen is continuous.

[0036] In one possible implementation, the first operation is a click operation on an album card. Before displaying the first grid interface, the method may further include: loading a first view control (referred to as a high-density view) and a second view control (referred to as a low-density view); initializing the size of the first view control and the number of columns for each position corresponding to the first view control, for example, the size of the first view control is 3100*7200 pixels wide, the corresponding month view has 13 columns, and the corresponding year view has 31 columns; initializing the size of the second view control and the number of columns for each position corresponding to the second view control, for example, the size of the second view control is 2170*6076 pixels, and the corresponding day view has 3 and 5 columns. The first grid interface is drawn based on the second view control, and the second grid interface is drawn based on the first view control. It should be noted that this embodiment of the application uses the example of a low-density view including two positions of day view, and a high-density view including one position of month view and one position of year view. In practice, a day view can include one, three, or more increments, while a month view and a year view can include two or more increments. Furthermore, the number of columns per increment in low-density and high-density views can be adjusted based on screen width, window width, and other factors.

[0037] In one possible implementation, the second view control inherits from the RecyclerView control, and the first view control inherits from the Android View control. Both the first and second view controls are larger than or equal to the screen size. The RecyclerView control has an ItemView that supports user touch input, while the Android View control does not. The RecyclerView control's ItemView supports user touch input on individual ItemViews. For lower-density views (such as day views), where users typically require grid-based touch input, RecyclerView is used as a low-density View. The Android View control is a standard View control, which can be considered a blank canvas, offering advantages in performance and memory efficiency. However, the Android View control lacks the concept of ItemView and does not support user touch input on individual ItemViews. For higher-density views (such as month or year views), typically used for quickly viewing multiple photos, where users generally do not require grid-based touch input, the Android View control can be used as a high-density View to save performance and memory.

[0038] In one possible implementation, the first view control supports at least two levels, with N columns and S columns respectively. The first grid interface can include P rows of grids, and the second grid interface can include Q rows of grids, where P and Q are positive integers. After displaying the first grid interface, the method may further include: obtaining a second stitched image based on the adaptation position values ​​of the images in the first grid interface, wherein each row of the second stitched image can include 2S-1 images, each column of the second stitched image can include T*2-(M*P / S) images, the adaptation position values ​​of adjacent grids in each row of the second stitched image differ by 1, and the adaptation position value of the image in the j-th row and i-th grid of the first stitched image differs from the adaptation position value of the image in the (j+1)-th row and i-th grid by S, where S is an integer greater than N and T is an integer greater than Q.

[0039] In the above scheme, when the photo album is opened, the electronic device first displays the daily view with fewer grid columns, loads the first stitched image corresponding to the monthly view and the second stitched image corresponding to the yearly view, so that the first stitched image and the second stitched image can be used directly when displaying the monthly view with 13 levels and the yearly view with 31 levels.

[0040] In one possible implementation, after displaying the second grid interface corresponding to the first image, the method may further include: receiving a third operation from the user on the electronic device, wherein the user's two fingers do not leave the screen from the second operation to the third operation; in response to the third operation, cropping the second image from the second spliced ​​image and displaying the fourth grid interface corresponding to the second image, wherein each row of the second image may include S images, each column of the second image may include T images, and the fourth grid interface may include a grid of S columns and T rows.

[0041] In the above scheme, the first and second stitched images are loaded when the electronic device enters the photo album. Upon receiving the second operation, a second grid interface based on the first stitched image is first displayed. While the user's two fingers remain on the screen, a third operation is received, displaying a fourth grid interface based on the second stitched image. When displaying the daily view, since the number of images to be displayed is small, the number of images the GPU needs to render is also small. Using an image stitching scheme would result in minimal or no reduction in overall processing time. Therefore, a grid interface rendering scheme based on low-density views and non-stitched images is adopted. When displaying the monthly and yearly views, the number of images the GPU needs to render is large. A grid interface rendering scheme based on high-density stitched images is adopted. By stitching multiple images, the number of image data and the number of times the GPU renders images can be reduced by orders of magnitude, improving the GPU's processing speed and making the photo album interface display smoother.

[0042] In one possible implementation, each grid cell in the first grid interface displays one image. Taking a daily view with 5 levels in the first grid interface as an example, the total number of grid cells in a daily view is relatively small, and the effect of using an image stitching scheme to shorten the processing time is not significant. In this case, there is no need to use the image stitching scheme of this application; instead, an original image is directly drawn for each grid cell.

[0043] In one possible implementation, each grid cell in the first grid interface displays a third image, which is a cropped image from a third stitched image. Each row of the third image can include M images, and each column can include P images. Each row of the third stitched image can include 2M-1 images. The adaptation position values ​​of adjacent cells in each row of the third stitched image differ by 1. The adaptation position value of the image in the j-th row and i-th cell of the third stitched image differs by M from the adaptation position value of the image in the (j+1)-th row and i-th cell. Taking a 13-level moon view as an example, the total number of cells in the moon view is relatively large. Although the image stitching process increases the time consumption, the GPU's image rendering process significantly reduces the time consumption, thus significantly shortening the overall time consumption. In this case, the image stitching scheme of this application can be adopted.

[0044] In one possible implementation, after displaying the second grid interface corresponding to the first image, the method may further include: obtaining a fourth stitched image based on the adaptation position values ​​of the images in the second grid interface, wherein each row of the fourth stitched image may include 2K-1 images, the adaptation position values ​​of adjacent grids in each row of the fourth stitched image differ by 1, and the adaptation position value of the image in the j-th row and i-th grid of the fourth stitched image differs from the adaptation position value of the image in the (j+1)-th row and i-th grid by K, where K is not equal to N.

[0045] In the above scheme, since the AP value of each grid is updated in the second grid interface, the AP range of the images that may be used when drawing the target grid interface (including K columns of grid) can be predicted again, and the images within the AP range are stitched together to obtain the fourth stitched image. This makes it easier to crop a large image from the fourth stitched image based on the scaling center when switching from the current gear to the target gear, and draw this large image into the View control.

[0046] Secondly, this application provides an apparatus comprising units for performing the method described in the first aspect above. This apparatus is adapted to perform the method described in the first aspect above, and a description of the units within this apparatus is provided in the description of the first aspect above; for brevity, it will not be repeated here.

[0047] The method described in the first aspect above can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, a processing module or unit, a display module or unit, etc.

[0048] Thirdly, this application provides an electronic device comprising: one or more processors, and a memory. The memory is coupled to the one or more processors and is used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the electronic device to perform the methods provided by the first aspect and any possible implementation thereof.

[0049] Fourthly, this application provides a computer-readable storage medium. The computer-readable storage medium includes computer instructions. When executed on an electronic device, the computer instructions cause the electronic device to perform the method provided by the first aspect and any possible implementation thereof.

[0050] Fifthly, this application provides a computer program product. When the computer program product is run on a computer, it causes the computer to perform the method provided by the first aspect and any possible implementation thereof.

[0051] Sixthly, this application provides a chip system applied to an electronic device, the chip system including one or more processors, the one or more processors being configured to invoke computer instructions to cause the electronic device to perform the methods provided in the first aspect and any possible implementation thereof.

[0052] It is understood that the beneficial effects achieved by the apparatus of the second aspect, the electronic device of the third aspect, the computer-readable storage medium of the fourth aspect, the computer program product of the fifth aspect, and the chip system of the sixth aspect provided above can be referred to as the beneficial effects of the first aspect and any possible implementation thereof, which will not be repeated here. Attached Figure Description

[0053] Figure 1 One of the schematic diagrams illustrating a scenario where a mobile phone displays images in a zoomed-out format on a photo album page, as provided in this application;

[0054] Figure 2 The second illustration of a scenario where a mobile phone displays images in a zoomed-out format on a photo album page, as provided in this application;

[0055] Figure 3 One of the schematic diagrams illustrating a scenario where a mobile phone displays images scrolling through a photo album page, as provided in this application;

[0056] Figure 4 The second illustration of a scenario where images are scrolling through a photo album page on a mobile phone, as provided in this application;

[0057] Figure 5 A schematic diagram of the View provided in this application for setting up an album page;

[0058] Figure 6 A schematic diagram illustrating the switching of the album page from 3 levels to 5 levels provided in this embodiment of the application;

[0059] Figure 7 One of the schematic diagrams of AP arrangement in 5 levels provided in the embodiments of this application;

[0060] Figure 8 This is a second schematic diagram of the AP arrangement in 5 levels provided in the embodiments of this application;

[0061] Figure 9 The third schematic diagram of AP arrangement in 5 levels provided in the embodiments of this application;

[0062] Figure 10 A schematic diagram illustrating a conventional grid-based interface drawing scheme provided in an embodiment of this application;

[0063] Figure 11 This is a schematic diagram of images to be stitched loaded in a high-density data loader, as provided in an embodiment of this application.

[0064] Figure 12 This is a schematic diagram of a stitched image loaded in a high-density bitmap loader, as provided in an embodiment of this application.

[0065] Figure 13 A schematic diagram illustrating a grid interface drawn based on spliced ​​images, provided in an embodiment of this application;

[0066] Figure 14 A schematic diagram illustrating the determination of the number of rows in a stitched image, provided in an embodiment of this application;

[0067] Figure 15 A schematic diagram illustrating the determination of the number of columns in the stitched images provided in an embodiment of this application;

[0068] Figure 16 One of the schematic diagrams of stitched images when switching from gear 5 to gear 3 according to an embodiment of this application;

[0069] Figure 17 This is the second schematic diagram of the stitched images provided in the embodiments of this application when switching from gear 5 to gear 3;

[0070] Figure 18 The third illustration of stitched images when switching from gear 5 to gear 3, provided as an embodiment of this application;

[0071] Figure 19 A software architecture diagram of an electronic device provided in an embodiment of this application;

[0072] Figure 20 A schematic diagram of a low-density view provided in this application;

[0073] Figure 21 A schematic diagram of another low-density view provided in this application;

[0074] Figure 22 A schematic diagram of a high-density view provided in this application;

[0075] Figure 23 A schematic diagram of another high-density view provided in this application;

[0076] Figure 24 A schematic diagram illustrating the switching between low-density and high-density views provided in this application;

[0077] Figure 25 A schematic diagram of LP and AP in a grid interface provided for this application;

[0078] Figure 26 A flowchart illustrating a method for switching from a low-density view to a high-density view, as provided in an embodiment of this application.

[0079] Figure 27A schematic diagram illustrating a method for reloading stitched images after scrolling through a photo album page based on a high-density View, as provided in an embodiment of this application.

[0080] Figure 28 A schematic diagram illustrating a scenario for an image scrolling scheme based on a View control, provided in an embodiment of this application;

[0081] Figure 29 A schematic diagram illustrating a method for drawing a month view based on a stitched image prepared when displaying a year view, provided in an embodiment of this application, when a user reduces the distance between two fingers;

[0082] Figure 30 A flowchart illustrating a specific scheme for determining the AP range based on the scaling center, as provided in this application embodiment;

[0083] Figure 31 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application.

[0085] In this embodiment of the application, the gallery application supports displaying images. For example, the gallery application supports displaying images in the form of albums on the album page, wherein the album page may include various types of albums such as camera, video, photo, cloud storage, selfie, live photo, long exposure, slow motion, screenshot and GIF.

[0086] For photo album pages, electronic devices have multiple preset levels. A "level" can be understood as the number of image columns displayed on the screen. For example, electronic devices may have preset levels of 3, 5, 13, and 31 for photo album pages. Level 3 is the minimum level, and level 31 is the maximum. At level 3, 3 images are displayed on the screen; at level 5, 5 images are displayed; at level 13, 13 images are displayed; and at level 31, 31 images are displayed.

[0087] Users can use a two-finger zoom gesture to change the number of image columns displayed on the screen for a photo album page. As an example, by decreasing the distance between two fingers, the user can trigger the electronic device to increase the number of image columns displayed on the screen for a photo album page. Conversely, by increasing the distance between two fingers, the user can trigger the electronic device to decrease the number of image columns displayed on the screen for a photo album page. For instance, if the user sets the photo album page to level 3 using a zoom gesture, the electronic device will maintain a display of 3 columns of images until the next two-finger gesture is received.

[0088] The following example uses a mobile phone as an electronic device, combined with... Figures 1 to 4 This section provides examples illustrating the use cases of image library applications.

[0089] In some embodiments, the phone supports zooming in and out of images on the photo album page in response to two-finger gestures.

[0090] For example, Figure 1 and Figure 2 This illustration shows a scenario where a mobile phone zooms in to display images on a photo album page.

[0091] Figure 1 and Figure 2 The example provided uses 3, 5, 13, and 31 preset levels for the photo album page, which is not intended to limit this application. In actual implementation, the phone can arbitrarily set the levels for the photo album page. For example, based on the phone's screen size, 1, 3, 5, 13, 21, and 31 levels can be preset for the photo album page, with level 1 being the minimum and level 31 being the maximum. Users can switch between these six levels by using a two-finger zoom gesture.

[0092] like Figure 1 As shown in (a), the phone displays icons for applications such as the Gallery app on the home screen. When a user wants to view images, they can tap the Gallery icon 01. In response to the user's tap on the Gallery icon 01, the phone displays... Figure 1 The card page is shown in (b). The card page comprises several areas: a status bar displaying battery and signal strength indicators, an action bar providing photos and a search box, and a foot bar showcasing photos, albums, memories, and creations. When an album in the foot bar is selected, the central area of ​​the card page displays album cards for the camera, all photos, and videos. These album cards, also known as photo album cards, each serve as an entry point to a specific album.

[0093] Users can select an album card from these album cards. For example, ... Figure 1As shown in (b), the user can tap the "Camera" card 02. In response to the user's tap on the "Camera" card 02, the phone displays the following... Figure 1 The album page shown in (c) is an example. The album page comprises multiple square grids. Taking an album page with three grids as an example, the side length h1 of each grid is equal to the width of the display screen divided by 3. Each grid displays a single image, which can be a thumbnail of a photograph or a video clip. Additionally, when the album page has three grids, a time capsule 03, displayed in days with preset transparency, is shown in the upper left corner of the album page; at this point, the album page is called a day view. As an example, the time capsule provided in this embodiment is a control or prompt that floats above the grid interface, indicating the shooting date of the images in the grid interface. For example, time capsule 03 indicates the specific shooting date "2024.07.10," which could be the shooting date of all photos in the current album page. When a user wants to quickly browse more photos, the user can reduce the distance between two fingers to trigger the phone to increase the number of image columns displayed on the album page.

[0094] like Figure 1 As shown in (d), the user can press two fingers, at which point the distance between the two fingers is d1. Figure 1 As shown in (e), without lifting two fingers from the screen, the user can decrease the distance between their fingers. When the distance decreases from d1 to d2, the album page switches from a 3-column grid to a 5-column grid, with each grid having a side length of h2, which is equal to the width of the screen divided by 5. When the album page includes 5 columns, a time capsule 04, displayed in days, is shown in the upper left corner with preset transparency. For example, the specific shooting date indicated by time capsule 04 is still "2024.07.10", which could be the shooting date of all photos in the current album page. At this point, the album page is still called the day view.

[0095] like Figure 1 As shown in (f), without lifting two fingers from the screen, the user can continue to decrease the distance between their fingers. When the distance decreases from d2 to d3, the album page switches from a 5-column grid to a 13-column grid, with each grid having a side length of h3, which is equal to the width of the screen divided by 13. When the album page includes 13 columns, two time capsules, 05 and 06, are displayed on the left side of the album page with preset transparency, each representing a month. The photos in the grid between time capsules 05 and 06 are dated "July 2024," while the photos in the rows containing time capsule 06 and below are dated "August 2024." At this point, the album page is called the Month View.

[0096] Users can further reduce the distance between their two fingers. For example, when the distance between the two fingers increases from... Figure 1 The d3 shown in (f) decreases to the following: Figure 2 As shown in (a) at d4, the album page switches from a 13-column grid to a 31-column grid, with each grid having a side length of h4, where h4 equals the width of the display divided by 31. When the album page includes 31 columns, a time capsule 07, displayed in years, is shown in the upper left corner of the album page with preset transparency. For example, time capsule 07 indicates a specific shooting date of "2024," which could be the shooting date of all photos in the current album page. At this point, the album page is called the year view.

[0097] As an example, after the album page is switched to a 31-column grid, such as Figure 2 As shown in (b), if the user raises their hand, the album page continues to display 31 columns of images.

[0098] As another example, after switching the album page to a 31-column grid, such as Figure 2 As shown in (c), if the user keeps the distance between their two fingers constant at d4, the album page will continue to display 31 columns of images.

[0099] As another example, after switching the album page to a 31-column grid, such as Figure 2 As shown in (d), if the user increases the spacing between two fingers, for example, from d4 to d5, the album page switches from a 31-column grid to a 13-column grid. When the album page includes 13 columns, two time capsules, 08 and 09, are displayed on the left side of the album page with preset transparency, each representing a month. The photos in the cell between Time Capsule 08 and Time Capsule 09 were taken on "July 2024," while the photos in the cells of Time Capsule 09 and the cells below it were taken on "August 2024." It can be understood that the user can continue to increase the spacing between two fingers to trigger the phone to increase the number of columns in the grid. For example, if the user continues to increase the spacing between two fingers, the album page switches from a 13-column grid to a 5-column grid; subsequently, if the user continues to increase the spacing between two fingers, the album page switches from a 5-column grid to a 3-column grid.

[0100] It should be noted that this application does not limit the number of image columns displayed when a mobile phone enters a photo album page. For example, the number of image columns displayed each time a mobile phone enters a photo album page can be as follows: Figure 1 The three columns shown in (c) can also be as follows: Figure 1 The five columns shown in (e) are an example. For instance, the number of image columns when the phone enters the album page this time is the same as the number of image columns when it exits the album page last time.

[0101] In other embodiments, the phone also supports scrolling images in response to a single finger gesture on the photo album page.

[0102] For example, Figure 3 and Figure 4 This illustration shows a scenario where a mobile phone scrolls through images on a photo album page.

[0103] Users can swipe down to trigger the phone to display more images taken earlier. For example... Figure 3 As shown in (a), the album page includes two time capsules, 10 and 11, organized by month. Time capsule 10 indicates the specific shooting date as "July 2024," and time capsule 11 indicates the specific shooting date as "August 2024." Users can press and hold image 455 with a single finger and then swipe down. Figure 3 As shown in (b), if a single finger slides down a distance d6, in response to the user's swipe, the album page slides down a distance d6. If all photos on the current page were taken in July 2024, then the specific shooting date indicated by Time Capsule 12 is "July 2024". Figure 3 As shown in (c), the user can continue to press and hold image 455 and slide it down a distance d7. In response to the user's sliding action, the album page slides down a distance d7. If all photos on the current page were taken in June 2024, then the specific shooting date indicated by Time Capsule 13 is "2024.06". Typically, the sliding distance of the downward follow-up sliding action is equal to the distance the album page scrolls down. The longer the downward follow-up sliding action, the more content is displayed when the album page scrolls down.

[0104] Users can also swipe up to trigger the phone to display more images taken later. For example... Figure 4 As shown in (a), the specific shooting date indicated by Time Capsule 14 on the album page is "2024.02". Users can press and hold image 350 with a single finger and then slide the finger upwards. (As shown in the image). Figure 4 As shown in (b), if a single finger slides upwards a distance d8, in response to the user's sliding action, the album page slides upwards a distance d8. If all photos on the current page were taken in March 2024, then the specific shooting date indicated by Time Capsule 15 will be "March 2024". Figure 4As shown in (c), the user can continue to press and hold image 350 and slide it upwards a distance d9. In response to the user's swipe, the album page slides upwards a distance d9. At this point, the current page includes two time capsules, 16 and 17, each representing a month. The photos in the cell between time capsules 16 and 17 were taken on "March 2024," while the photos in the cell containing time capsule 17 and the cells below it were taken on "April 2024." Typically, the upward swipe distance is equal to the distance the album page scrolls upwards. The longer the upward swipe distance, the more content is displayed when the album page scrolls upwards.

[0105] In the above embodiments, the album page includes multiple grids, each displaying one image. An image is also referred to as a texture image. It should be noted that, due to size limitations, this application embodiment only shows the texture image in a grid with three levels of resolution. It is understood that as the resolution increases, each grid may include a smaller texture image. To display multiple images on the album page, this application provides a drawing scheme for a grid interface based on a View control. The View control's size is greater than or equal to the screen size and can be considered a container for displaying images.

[0106] For example, Figure 5 This diagram illustrates the View control settings for a photo album page.

[0107] After the phone receives a click on the Gallery icon, it starts the Gallery app and initializes a View control. The size of the View control is greater than or equal to the screen / display window size. For example, with a screen size of 1300 pixels wide x 3470 pixels high, the phone divides the screen width of 1300 pixels by the minimum grid size of 3, resulting in a side length of 434 pixels for each grid cell. Then, the phone divides the screen height of 3470 pixels by the side length of each grid cell of 434 pixels to obtain the number of rows in the drawing area: 8. Figure 5As shown, at level 3, the grid interface drawn in the View control's drawing area is a 3-column * 8-row grid. Each grid displays one image. Each image has a unique index, such as the adapter position (AP). The AP values ​​of each grid in the drawing area are arranged according to the first rule: starting from the grid in the first row and first column of the drawing area, the AP values ​​of each grid increase row by row and column by column. The AP values ​​of adjacent grids in a row differ by 1, and the AP value of the last grid in the j-th row differs by 1 from the AP value of the first grid in the (j+1)-th row, where j is a positive integer. The image displayed in each grid can be a thumbnail of a photograph or a thumbnail of a frame from a video. When an image is a thumbnail of a photograph, clicking the thumbnail will display the photograph in the display window. When an image is a thumbnail of a frame from a video, clicking the thumbnail will play the video in the display window.

[0108] In this embodiment, when a user presses two fingers on the screen, the electronic device calculates a zoom center based on the coordinates of the two fingers. The user can then change the distance between the two fingers, and the electronic device switches zoom levels in response to the two-finger zoom operation. Figure 6 As shown, when the user reduces the distance between two fingers, the album page switches from level 3 to level 5. When switching levels, the electronic device can keep the AP value of the cell where the zoom center is located unchanged and rearrange the AP values ​​of other cells according to the first rule mentioned above.

[0109] For example, Figures 7 to 9 This diagram illustrates the AP arrangement at level 5 when the zoom center is changed.

[0110] like Figure 7 As shown, if the AP value of the cell where the zoom center is located is 110, then when switching from level 3 to level 5, the electronic device can maintain the AP value of the cell where the zoom center is located at 110, and arrange the AP values ​​of other cells based on the AP value of 110. From the cell in the first row and first column of the drawing area to the cell in the last row and last column, the AP values ​​are 83, 84, 85, ..., 152 respectively.

[0111] like Figure 8 As shown, if the AP value of the cell where the zoom center is located is 100, then when switching from level 3 to level 5, the electronic device can maintain the AP value of the cell where the zoom center is located at 100, and arrange the AP values ​​of other cells based on the AP value of 100. From the cell in the first row and first column of the drawing area to the cell in the last row and last column, the AP values ​​are 100, 101, 102, ..., 169 respectively.

[0112] like Figure 9As shown, if the AP value of the cell where the zoom center is located is 123, then when switching from level 3 to level 5, the electronic device can maintain the AP value of the cell where the zoom center is located at 123, and arrange the AP values ​​of other cells based on the AP value of 123. From the cell in the first row and first column of the drawing area to the cell in the last row and last column, the AP values ​​are 54, 55, 56, ..., 123.

[0113] According to the above embodiments, Figures 7 to 9 As described, when the zoom center is different, the specific position of the grid cell containing the zoom center in the display window will change, and the AP value of the grid cell will also change. This will cause the electronic device to rearrange the grid cells according to the first rule, and the AP value will also change. Based on this, the electronic device needs to draw the image in each grid cell sequentially each time the zoom level is switched. Specifically, the CPU calls the ondraw() method to send drawing instructions (also known as drawing tasks) to the GPU.

[0114] Taking the 31st level as an example, the drawing area in the View control includes 31 columns * 72 rows, totaling 2232 grids. Figure 10 As shown, the CPU can create a View control larger than the screen size. When the user triggers a zoom operation to switch the electronic device from column 1 to column 31, the CPU can send the drawing task of 2232 images to the GPU. The GPU reads images 1 to 2232 from the data cache module and draws the read images sequentially into the respective grids of the drawing area. For 2232 images, the GPU performs a total of 2232 drawing processes. Experimental data shows that the time from the CPU sending the drawing task to the GPU to the GPU sending the image to the screen is approximately 35ms.

[0115] Relevant data requires that when the display refresh rate is 120 Hz and the device is under load, the rendering time of the grid interface must be controlled within a preset duration (e.g., 8ms). When there are many images to be displayed in the grid interface, the CPU will issue a large number of rendering instructions to the GPU. This requires the GPU to render a large number of images, resulting in a longer rendering time. For example, rendering 2232 images takes approximately 35ms, which far exceeds the required 8ms. This can cause stuttering or frame drops in the grid interface, or blank areas in the grid, affecting the user experience.

[0116] In view of the above problems, this application provides a drawing scheme for a grid interface based on stitched images. In this scheme, the electronic device can predict the range of images that may be used when drawing the target grid interface (including Q rows and N columns, where N and Q are positive integers) based on the AP value of the image in the first grid (i.e., the first row and first column) and the AP value of the image in the last grid (i.e., the last row and last column) of the current grid interface. Then, it stitches the images within the AP range according to a preset rule to obtain a stitched image. Here, one row includes 2N-1 images, the AP values ​​of adjacent grids in one row differ by 1, and the AP value of the image in the j-th row and i-th grid differs from the AP value of the image in the (j+1)-th row and i-th grid by N, where i and j are positive integers. When switching from the current gear to the target gear, the CPU crops a large image from the stitched image based on the scaling center, and the GPU responds to the drawing instructions from the CPU by drawing this large image onto the View control.

[0117] In traditional grid-based interface rendering schemes, if there are N*Q images to be displayed in a single frame, the GPU renders N*Q images and performs N*Q rendering operations. However, in the grid-based interface rendering scheme based on image stitching provided in this application, if there are N*Q images to be displayed in a single frame, by cropping a larger image corresponding to each of the N*Q images from the stitched image, the number of images rendered by the GPU can be reduced from N*Q to 1, and the number of rendering operations can be reduced to 1, thereby improving GPU processing speed. It can be understood that the larger the value of N*Q, the more significant the speed improvement in the image stitching-based grid-based interface rendering scheme provided in this application, compared to the traditional scheme.

[0118] In the grid-based interface drawing scheme based on image stitching provided in this application, an image to be displayed is referred to as an image to be stitched. The image attributes of the image to be stitched and the stitched image obtained from the image to be stitched can be bitmaps.

[0119] For example, Figure 11 This is a schematic diagram of the images to be stitched loaded in the high-density data loader provided in this application. Accordingly, Figure 12 This is a schematic diagram of a stitched image loaded in a high-density bitmap loader, as provided in this application.

[0120] Electronic devices can pre-load images to be stitched into a high-density data loader based on the current application scenario. For example, from a photo album page... Figures 7 to 9Before switching from a 3-column daily view to a 5-column (N=5) daily view, the electronic device can trigger a high-density data loader to load the images to be stitched based on the AP values ​​of the images in the first and last cells of the 3-column daily view. For example... Figure 11 As shown, the AP values ​​of the images to be stitched loaded in the high-density data loader are "53", "54", "55", ... . Each image to be stitched has a size of 100 pixels wide * 100 pixels high. A smaller AP value indicates that the electronic device acquired the image earlier, and a larger AP value indicates that the electronic device acquired the image later. Here, "image acquisition time" refers to the time when the original image or video was stored in the album. This original image or video was taken with a camera, downloaded from a network device, or obtained through a screenshot. The image to be stitched refers to a thumbnail of the original image or video.

[0121] After the high-density data loader finishes loading the images to be stitched, such as Figure 12 As shown, the CPU can stitch together every 2N-1 adjacent / connected images in ascending order of AP values, and then continue stitching together the images in the next row. The AP value of the image in the i-th cell of the j-th row differs from the AP value of the image in the i-th cell of the (j+1)-th row by N, where i and j are both positive integers. Taking N=5 as an example, the electronic device can stitch together 9 images with AP values ​​of 53, 54, ..., 61 in the first row; 9 images with AP values ​​of 58, 59, ..., 66 in the second row; 9 images with AP values ​​of 63, 64, ..., 71 in the third row... and so on, stitching together 9 images with AP values ​​of 163, 164, ..., 171 in the 23rd row, finally obtaining a stitched image with a resolution of 900 pixels wide * 2300 pixels high. Thus, for a 5-column daily view, the number of images drawn by the GPU is reduced from 5*14=70 to 1.

[0122] like Figure 12 The stitched image shown covers the drawing as follows Figures 7 to 9 The large image used when showing the 5-column daily view is as follows. Figure 12 As shown, the clipping region enclosed by line 1 can be used to draw, for example... Figure 8 The 5-column day view shown, with the cropped area enclosed by line type 2, can be used to draw, as... Figure 9 The 5-column day view shown, with the cropped area enclosed by line type 3, can be used to draw, for example... Figure 7 The daytime view shown has 5 columns. (See attached image.) Figure 8 As shown, when scaling with AP=100 as the scaling center, the AP=123 in the last row and last column of the grid interface is related to the maximum AP value of AP=169 when the target grid interface is drawn; as Figure 9As shown, when scaling with AP=123 as the scaling center, the AP=100 in the first row and first column of the grid interface is related to the minimum AP range of AP=54 when drawing the target grid interface. It can be understood that, in situations like... Figure 12 The stitched images shown include those used for drawing, such as Figure 8 and Figure 9 In the case of a large view of the daytime chart with 5 columns as shown, if the zoom center is located at, for example... Figures 7 to 9 If any one of the three columns in the daytime view shown is selected, then the electronic device can be accessed from, for example... Figure 12 The corresponding large image is cropped from the stitched image shown, and a 5-column daily view is drawn based on the large image.

[0123] For example Figure 10 The traditional grid-based interface shown is... Figure 13 A schematic diagram of a grid interface drawn based on stitched images is provided. When an electronic device displays a 13-column month view, the device's high-density bitmap loader can predict the AP range of images that might be used when drawing the year view based on the AP values ​​of the images in the first and last grids of the month view. Then, it stitches the images within the AP range according to preset rules to obtain the desired result. Figure 13 The image shown is a stitched image. When a user triggers the electronic device to zoom from a 13-column month view to a 31-column year view, the CPU can crop a larger image from the stitched image based on the zoom center. The GPU, responding to the drawing command from the CPU, retrieves the larger image from the high-density bitmap loader and draws it onto the View control. Experimental data shows that the time from the CPU issuing the drawing task to the GPU sending it to the display is approximately 35ms.

[0124] It should be noted that, Figures 7 to 9 The examples provided are 3-column day view with 24 grids and 5-column day view with 70 grids, and do not limit the scope of this application. The size and number of grids in each view are determined by the screen size. When the number of grids in the 3-column day view is changed, the number of grids in the 5-column day view will also change accordingly.

[0125] To better understand the grid interface drawing scheme based on stitched images provided in this application, the following example illustrates how, when switching from the current grid interface to the target grid interface, the stitched image used to draw the target grid interface is calculated based on the number of rows and columns in the current grid interface. Here, there are no other grids between the target and current grids; that is, the target grid is the next level after the current grid, for example, the current grid is level 5 and the target grid is level 13. Alternatively, there is at least one grid between the target and current grids, for example, the current grid is level 5 and the target grid is level 31.

[0126] First, let's introduce the six parameters: curSpanCount, targetSpanCount, curRowCount, targetRowCount, canvasSpanCount, and canvasRowCount. Here, curSpanCount represents the number of columns in the grid interface for the current gear, targetSpanCount represents the number of columns in the grid interface for the target gear, curRowCount represents the number of rows in the grid interface for the current gear, targetRowCount represents the number of rows in the grid interface for the target gear, canvasSpanCount represents the number of columns in the stitched image, and canvasRowCount represents the number of rows in the stitched image.

[0127] (a) When switching from a smaller gear with fewer grid columns to a larger gear with more grid columns, the following relationship must be satisfied:

[0128] canvasSpanCount=targetSpanCount*2-1;

[0129] canvasRowCount=targetRowCount*2-(curSpanCount*curRowCount / targetSpanCount).

[0130] For example, in a lower gear Figures 7 to 9 The three-column grid interface shown has a large selection as follows: Figures 7 to 9 Taking the 5-column grid interface shown as an example, curSpanCount = 3 columns, targetSpanCount = 5 columns, curRowCount = 8 rows, and targetRowCount = 14 rows.

[0131] When shifting from 3rd gear to 5th gear, such as Figures 7 to 9As shown, in a 5-level grid interface, the following conditions must be met: the AP values ​​of adjacent grids in a row differ by 1, and the AP value of the image in the i-th grid of the j-th row differs from the AP value of the image in the i-th grid of the (j+1)-th row by targetSpanCount, where i and j are positive integers.

[0132] When the zoom center is any cell in the small-scale grid interface, in order to crop the large image corresponding to the cell where the zoom center is located from the stitched image, the following must be satisfied: when any AP value of the target scale is used as the AP value of the cell in the first row and first column of the large image, there must be 4 cells to the right of this AP value, corresponding to targetSpanCount-1 = 4 cells. Based on this, as follows: Figure 14 As shown, canvasSpanCount = targetSpanCount * 2 - 1 = 5 * 2 - 1 = 9 columns.

[0133] Specifically, column 5 is the middle column of the stitched images, and satisfies the following conditions: the AP values ​​of adjacent cells in row 1 differ by 1, and the AP value of the image in cell i of row j differs from the AP value of the image in cell i of row (j+1) by targetSpanCount = 5. Thus, the AP value of the image in cell i of row (j+1) is equal to the AP value of the image in cell i+targetSpanCount of row j. For example, the AP value of cell i in row 4, column 3 is 70, and the AP value of cell i in row 3, column 8 is 70; the AP value of cell i in row 10, column 4 is 101, and the AP value of cell i in row 9, column 9 is 101. In other words, in the stitched image, besides the AP value in the middle column, all other AP values ​​will appear once on each side of the middle column. This ensures that when any predicted AP value is used as the AP value for the first cell in the first row and first column of the larger image, there will be four cells to the right of that AP value in the stitched image. This allows the electronic device to crop a larger image with five columns, where the AP values ​​increase sequentially row by row and column by column. For example, if AP = 54 is used as the AP value for the first cell in the first row and first column of the larger image, the AP values ​​to its right will be 55, 56, 57, and 58. Similarly, if AP = 83 is used, the AP values ​​to its right will be 84, 85, 86, and 87. And if AP = 100 is used, the AP values ​​to its right will be 101, 102, 103, and 104.

[0134] When the zoom center is any cell in the small-grid interface, in order to crop the large image corresponding to the cell where the zoom center is located from the stitched image, the following conditions must be met: the stitched image must include at least the large image with the first row and first column of the current grid interface as the zoom center, and the large image with the last row and last column of the current grid interface as the zoom center. It can be understood that when switching from a small-grid interface with fewer columns to a large-grid interface with more columns, the number of cells in the grid interface increases. The first row and first column of the current grid interface involves the maximum value of the AP range in the stitched image, and the last row and last column of the current grid interface involves the minimum value of the AP range in the stitched image. Therefore, when the stitched image includes the large image with the first row and first column of the current grid interface as the zoom center, and the large image with the last row and last column of the current grid interface as the zoom center, if the zoom center is located in any cell in the current grid interface, then the electronic device can crop the corresponding large image from the stitched image. like Figure 15 As shown, Okay. Among them, The rounding up symbol.

[0135] Specifically, when shifting from 3rd gear to 5th gear, such as Figure 15 As shown, the cropping area enclosed by line 1 is used to draw the grid interface when zooming in with AP=100 as the zoom center, and the cropping area enclosed by line 2 is used to draw the grid interface when zooming in with AP=123 as the zoom center. Both grid interfaces consist of 14 rows, with 5 overlapping rows. After removing the overlapping rows, the final number of rows for the stitched image is determined to be 23.

[0136] (ii) When switching from a gear with more grid columns to a gear with fewer grid columns, the following relationship must be satisfied:

[0137] canvasSpanCount=targetSpanCount*2-1;

[0138] canvasRowCount=curSpanCount*curRowCount / targetSpanCount.

[0139] When shifting from 5th gear to 3rd gear, such as Figures 16 to 18 As shown, in a 3-level grid interface, the following conditions must be met: the AP values ​​of adjacent grids in a row differ by 1, and the AP value of the image in the i-th grid of the j-th row differs from the AP value of the image in the i-th grid of the (j+1)-th row by targetSpanCount, where i and j are positive integers.

[0140] When the zoom center is any cell in the large-scale grid interface, in order to crop the large image corresponding to the cell where the zoom center is located from the stitched image, the following must be satisfied: when any AP value of the target scale is used as the AP value of the cell in the first row and first column of the large image, there must be two cells to the right of this AP value: targetSpanCount-1 = 2. Based on this, as follows: Figures 16 to 18 As shown, canvasSpanCount = targetSpanCount * 2 - 1 = 3 * 2 - 1 = 5 columns.

[0141] Specifically, the third column is the middle column of the stitched image, and it satisfies the following conditions: the AP values ​​of adjacent cells in row 1 differ by 1, and the AP value of the image in the i-th cell of row j differs from the AP value of the image in the i-th cell of row (j+1) by targetSpanCount = 3. Thus, the AP value of the image in the i-th cell of row (j+1) is equal to the AP value of the image in the i+targetSpanCount cell of row j. In other words, in the stitched image, except for the AP values ​​in the middle column, all other AP values ​​must appear once on the left and once on the right of the middle column. This ensures that when any predicted AP value is used as the AP value of the cell in the first row and first column of the large image, there are two cells to the right of this AP value in the stitched image. Therefore, the electronic device can crop a large image with 3 columns from the stitched image, where the AP value increases row by row.

[0142] When the zoom center is any cell in the large-scale grid interface, in order to crop the large image corresponding to the cell where the zoom center is located from the stitched image, the following conditions must be met: the stitched image must include at least the large image with the first row and first column of the current grid interface as the zoom center, and the large image with the last row and last column of the current grid interface as the zoom center. It can be understood that when switching from a large to a small scale, the number of cells in the grid interface decreases. The first row and first column of the current grid interface involves the minimum value of the AP range in the stitched image, and the last row and last column of the current grid interface involves the maximum value of the AP range in the stitched image. Therefore, when the stitched image includes the large image with the first row and first column of the current grid interface as the zoom center, and the large image with the last row and last column of the current grid interface as the zoom center, if the zoom center is located in any cell in the current grid interface, then the electronic device can crop the corresponding large image from the stitched image. For example... Figures 16 to 18 As shown,

[0143] in,

[0144] This is the floor symbol.

[0145] The above embodiments describe how to calculate the number of rows and columns of the stitched image when switching from the current gear to the target gear. After calculating the number of rows and columns of the stitched image, the electronic device also needs to set an AP value for each cell in the stitched image. Specifically, the electronic device can calculate the AP value of the center cell based on the AP values ​​of the first and last cells in the current gear; set this AP value as the AP value of the center cell of the stitched image, that is, the AP value of the center cell of the current gear and the AP value of the center cell of the stitched image are equal; then, according to the rule that the AP values ​​of adjacent cells in a row differ by 1, and the AP value of the i-th cell in the j-th row differs by N from the AP value of the i-th cell in the (j+1)-th row, the AP value of each cell in the stitched image is calculated. Here, the center cell refers to the cell located in the middle row and middle column of the day view, month view, or year view.

[0146] For example, if the AP of the first cell is denoted as startAP and the AP of the last cell is denoted as endAP, then the AP value of the center cell is represented as: (startAP + endAP1) / 2. For instance, if startAP = 100 and endAP = 123, then the AP value of the center cell is: or,

[0147] The following provides an example of a View control used to hold day, month, and year views.

[0148] The above-mentioned grid interface drawing scheme based on image stitching includes two processes: The first process is that the CPU predicts the range of images that may be used when drawing the target grid interface (including N columns) based on the AP value of the image in the first grid and the AP value of the image in the last grid in the current grid interface, and stitches the images within the AP range according to preset rules to obtain a stitched image, which is then stored in a high-density bitmap loader. Due to the addition of the image stitching process, it takes slightly more time than the traditional grid interface drawing method. The second process is that when switching from the current gear to the target gear, the CPU crops a large image from the stitched image based on the scaling center. The GPU responds to the drawing instructions from the CPU and draws this large image onto the View control, thereby reducing the number of images drawn by the GPU from N*Q to 1 and the number of times the image is drawn to 1. This process significantly shortens the time compared to the traditional grid interface drawing method.

[0149] For drawing scenarios where the number of images to be displayed is small, such as Figure 1 (c) or Figure 1As shown in (e) of the daytime view, the number of images that the GPU needs to draw is relatively small. The first process mentioned above increases the time consumption, while the second process mentioned above does not significantly reduce the time consumption. This results in the two processes not significantly reducing the overall time consumption.

[0150] For drawing scenarios where a large number of images need to be displayed, such as Figure 1 The lunar view shown in (f) or as shown in the figure Figure 2 As shown in (a) of the year view, the GPU needs to draw a large number of images. Although the first process mentioned above increases the time consumption, the second process mentioned above has a very obvious effect on shortening the time consumption, which leads to a significant reduction in the overall time consumption between the two processes.

[0151] In one possible use case, regardless of the number of columns of images in the grid interface, users have a need for touch interaction with the grid, such as selecting multiple original images by clicking on a grid and then combining the selected original images into a video. Based on this, the grid interface drawing scheme based on stitched images provided in this application can be applied to views at various levels, such as day, month, or year views. In this case, the aforementioned View control can be a custom RecyclerView control. The custom RecyclerView control inherits from the RecyclerView control. The RecyclerView control is a special type of View control composed of multiple item views in a grid format, each item view displaying an image view. Each item view supports responding to user touch operations on individual item views. The custom RecyclerView extends the functionality of the RecyclerView control, supporting scrolling and zooming. For example, when a user swipes their finger up or down on the screen, the custom RecyclerView scrolls accordingly, causing the grid interface to scroll up and down as well; or, when a user zooms in or out on the grid interface, the custom RecyclerView zooms in or out accordingly, causing the grid interface to zoom in or out as well. Each ItemView supports two ways of drawing controls: one is to display multiple controls in a multi-view layout; the other is to convert each view in the multi-view layout into a drawable object and place it in a drawable list, then trigger the drawing of each drawable individually.

[0152] In another possible use case, when the grid interface includes fewer columns of images (3 or 5 columns), users typically have a need to interact with the grid, such as clicking on a grid to trigger the display of the original photo or video corresponding to a thumbnail. When the grid interface includes more columns of images (13 or 31 columns), it is generally used for quickly browsing multiple photos, in which case users generally do not need to interact with the grid. Therefore, the grid interface drawing scheme based on stitched images provided in this application is only applicable to monthly or yearly views. For daily views, the following method still applies. Figure 10 The traditional grid-based interface drawing scheme is shown. In this case, the embodiments of this application specifically divide the above-mentioned View control into two types: low-density View and high-density View. Among them, the low-density View is implemented through a custom RecyclerView control and is used for low-level views (such as day view), supporting touch operation of individual ItemViews, as well as scrolling or scaling of the entire custom RecyclerView control; the high-density View is implemented through Android's View control and is used for high-level views (such as month view or year view), supporting scrolling or scaling of the entire custom RecyclerView control.

[0153] Based on the description above, the custom RecyclerView control's ItemView supports user touch operations on individual ItemViews. For lower-density views (such as day views), users generally have a need for grid-based touch controls, so a custom RecyclerView is used as the low-density view. The difference between Android-based View controls and custom RecyclerViews is that Android-based View controls are ordinary View controls, which can be considered a blank canvas, offering advantages in performance and memory efficiency. However, Android-based View controls do not have the concept of ItemViews and do not support user touch operations on individual ItemViews. For higher-density views (such as month or year views), which are generally used for quickly viewing multiple photos, users typically do not have a need for grid-based touch controls. To save performance and memory, Android-based View controls can be used as the high-density view.

[0154] For example, Figure 19 A software architecture diagram of an electronic device is shown.

[0155] like Figure 19As shown, electronic devices can adopt a layered architecture, dividing the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the software layers of the software architecture are divided from top to bottom as follows: application (APP) layer, application framework (FWK) layer, system library, and kernel layer. This software architecture runs on top of the hardware layer, which may include GPUs, displays, and sensors, etc.

[0156] The application layer can include a series of application packages, such as a camera app and a gallery app. The camera app can use the webcam to take photos and videos and store them in the gallery app. The gallery app can manage not only the captured photos and videos, but also screen captures and shared images. The gallery app can include a homepage control module, a grid control module, view controls, a low-density data loader, a high-density data loader, and a high-density bitmap loader. The view controls can include low-density views and high-density views. The low-density data loader stores the original, unstitched images prepared for low-density views. The high-density data loader (also known as the Item data cache module) stores the unstitched images prepared for high-density views. The high-density bitmap loader stores the stitched images prepared for high-density views; these stitched images are obtained by stitching together images within the storage range of the high-density data loader.

[0157] The specific functions of each module of the gallery application can be found in the description of the following embodiments, and will not be repeated here.

[0158] When these application packages are run, they can access the various service modules provided by the application framework layer through the application programming interface (API) and execute corresponding intelligent business operations.

[0159] The application framework layer provides APIs and a programming framework for applications. It includes predefined functions. For example, the application framework layer may include a window manager service (WMS), an activity manager service (AMS), SurfaceFlinger, and a file manager. Specifically, WMS is used to pass data collected by the touch driver to the currently bound window control, such as a RecyclerView or ReceiveTouchView control.

[0160] The system library can include multiple functional modules, such as the surface manager, media libraries, a 2D graphics engine (e.g., SGL), and a 3D graphics processing library (e.g., OpenGL ES). Within the system library, the Android Runtime includes the core libraries and the virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core Android libraries. The application layer and application framework layer run in the virtual machine.

[0161] The kernel layer is the layer between hardware and software, and belongs to the lower level of the Android system. The kernel layer can contain various driver interfaces, such as display drivers, audio drivers, touch drivers, and sensor drivers.

[0162] It should be noted that although the embodiments of this application are illustrated using the Android system as an example, the basic principles are equally applicable to electronic devices based on operating systems such as iOS or Windows.

[0163] To facilitate understanding, taking a photo album window with dimensions of 1300 pixels wide and 3470 pixels high (the photo album window is currently displayed in full screen) as an example, the following examples illustrate the specific implementation methods of low-density views and high-density views.

[0164] When a grid interface includes a limited number of images, such as 3 or 5 columns, electronic devices can display relevant feature icons (also known as controls) on the thumbnails based on the business status of the photos and videos. Figure 1 As shown in (c), the bottom right corner of the cloud photo grid includes a cloud control to indicate that the photo has been synced to the cloud; the top right corner of the favorited photo grid includes a favorite control to indicate that the photo has been favorited by the user. In some scenarios, electronic devices need to support user clicks on a control to achieve the corresponding function. However, Android's View control does not support touch operations on controls and therefore cannot achieve the corresponding function. For this reason, this application provides a grid interface display scheme based on a custom RecyclerView control (i.e., a low-density View). The custom RecyclerView control is larger than the screen size. The custom RecyclerView consists of multiple ItemViews in a grid format, each ItemView displaying an image view and supporting user touch operations on individual ItemViews.

[0165] For example, Figure 20 and Figure 21 This is a schematic diagram of a low-density View (implemented based on a custom RecyclerView control) provided in this application.

[0166] When the album window needs to display 3 or 5 columns of images, the total number of images in the album window is relatively small, so a single low-density view can be used. If the scaling factor of the low-density view is set to 1.0 when the album window displays 3 columns, then when the album window displays 5 columns, the low-density view needs to be shrunk by 3 / 5 = 0.6 times. Based on this, the size of the low-density view is calculated as follows: When the album window needs to display 3 columns of images, divide the width of the album window (1300 pixels) by 3 columns to get the width / height of each grid cell as 434 pixels; divide the height of the album window (3470 pixels) by the height of each grid cell as 434 pixels to get the number of rows of images that the album window needs to display as 8; then, multiply the width of each grid cell (434 pixels) by 5 columns to get the width of the low-density view as 2170 pixels; multiply the width of the low-density view (2170 pixels) by the height of the album window (3470 pixels), then divide by the width of the album window (1300 pixels), then divide by the height of each grid cell as 434 pixels to get the number of rows of the low-density view as 14. Thus, the low-density view consists of 5 columns * 14 rows of grid cells, with each grid cell having a width of 434 * a height of 434 pixels. Figure 20 As shown, when the scaling factor of the low-density view is set to 1.0, the daytime view drawing area (3 columns * 8 rows grid) of the low-density view is sent to the display screen with a scaling factor of 1.0. Figure 21 As shown, when the scaling factor of the low-density view is set to 0.6, the day view drawing area (5 columns * 14 rows grid) of the low-density view is sent to the display screen at a scaling factor of 0.6.

[0167] When a grid interface includes a large number of columns of images, such as 13 or 31 columns, the electronic device does not need to draw controls on the thumbnails. In this case, the electronic device does not need to support user clicks on any control to achieve the corresponding function. Android's View control can fully meet the display function of a grid interface, and the implementation is simpler. For this reason, this application provides a grid interface display solution based on Android's View control (i.e., high-density View). The high-density View extends the functionality of Android's View control, supporting scrolling and zooming, but it does not have the concept of ItemView.

[0168] For example, Figure 22 and Figure 23 This is a schematic diagram of a high-density View (implemented based on Android View controls) provided for this application.

[0169] When the photo album window needs to display 13 or 31 columns of images, the electronic device can create a high-density view with a size larger than or equal to that of the album window. For example, if the album window is 1300 x 3470 pixels wide, the high-density view would be 3100 x 7200 pixels wide. The high-density view can be considered a blank canvas. The electronic device can draw a grid interface within the high-density view based on the number of columns of images to be displayed. Figure 22 As shown, the electronic device can draw a grid interface (13 columns * 30 rows) in the moon view drawing area of ​​the high-density view, and send the grid interface to the display screen at a scaling factor of 1.0. Figure 23 As shown, the electronic device can draw a grid interface (31 columns * 72 rows) in the annual view drawing area of ​​the high-density view, and send the grid interface to the display screen according to the scaling factor 13 / 31 = 0.42.

[0170] During the display of the grid interface, the electronic device can respond to the user's zoom operation by switching between a low-density view and a high-density view to change the number of grid columns in the current grid interface.

[0171] For example, Figure 24 This is a schematic diagram illustrating the switching between a low-density view and a high-density view, as provided in this application.

[0172] For example, a low-density view can be like... Figure 20 and Figure 21 The low-density view shown can be a high-density view, such as... Figure 22 and Figure 23 The high-density view shown. (As shown in the image) Figure 24 As shown, when the user decreases the distance between their two fingers, the electronic device switches from a low-density view to a high-density view. The high-density view uses a grid-based interface rendering scheme based on Android's View controls, where images are stitched together. This is understandable because, in this scenario, the number of images to be displayed is large, thus significantly reducing the overall processing time. When the user increases the distance between their two fingers, the electronic device switches from a high-density view to a low-density view. The low-density view uses a grid-based interface rendering scheme based on a custom RecyclerView, where images are not stitched together. This is understandable because, in this scenario, the number of images to be displayed is small, thus reducing the processing time. Furthermore, since there is no need to stitch images, this scheme does not increase CPU power consumption compared to a grid-based interface rendering scheme based on stitched images.

[0173] The above embodiments introduce two types of Views: low-density View and high-density View. Both low-density and high-density Views involve two indices: one for the adapter position (AP) and the other for the layout position (LP).

[0174] In this context, AP is an index specifically set by the adapter for the content displayed on the screen, representing the number of the image displayed in the grid. LP is an index created by the LayoutManager, representing the position of the grid within the View control, such as the row and column number. In the native logic of the Android system, LP cannot be customized, but in this embodiment, the electronic device can change the specific value of LP based on scaling factors, scaling center, and scrolling operations. In a low-density View, LP is consecutively numbered from the first row and first column of the low-density View to the last row and last column of the low-density View. In a high-density View, LP is consecutively numbered from the first row and first column of the high-density View's drawing area to the last row and last column of the high-density View's drawing area.

[0175] For example, taking a low-density View as an example, Figure 25 A schematic diagram of LP and AP in a grid interface is shown.

[0176] For ease of understanding, let's take an example where the maximum density of a View is 5 levels, and the width and height of each ItemView are equal to the screen width divided by 3, meaning the minimum density is 3 levels. Figure 25 As shown in (a), before receiving a user's two-finger zoom operation, the low-density View actually displayed on the screen is shrunk to the same width as the screen, and the actual layout position index and actual adaptation position index of each ItemView are equal. For example, the ItemView located in the first row and first column at the top left corner of the screen has an actual layout position index LP = 0 and an actual adaptation position index AP = 0, while the ItemView located in the last row and last column at the bottom right corner of the screen has an actual layout position index LP = 49 and an actual adaptation position index AP = 49. When the user wants to view the magnified image of the grid (i.e., ItemView) enclosed by the dashed frame, the user can place two fingers within the area enclosed by the dashed frame and gradually increase the distance between the two fingers. In response to the two-finger zoom operation, the phone zooms in on the entire low-density View centered on the ItemView enclosed by the dashed frame. The ItemView displayed on the screen is updated to 3 columns.

[0177] like Figure 25 As shown in (b), the phone can renumber the actual adaptation position index values ​​corresponding to the ItemViews in columns 1 to 3 of the low-density View. The numbering rules for the actual adaptation position index values ​​corresponding to these ItemViews are as follows: the actual adaptation position index of the ItemView enclosed by the dashed box remains AP = 6; starting from the ItemView enclosed by the dashed box, the AP value of each ItemView before the ItemView enclosed by the dashed box is decremented by 1, and the AP value of each ItemView after the ItemView enclosed by the dashed box is incremented by 1. For the ItemViews in columns 4 to 5 of the low-density View, the phone can assign an AP value of -1 to each ItemView. Here, AP = -1 means that the actual adaptation position value is invalid. At this time, an ItemView can display a blank image or maintain the display as shown. Figure 10 The image shown illustrates the correspondence between the actual layout position index and the actual adaptation position index of an ItemView. When scaling a low-density View with the ItemView at layout position index LP=6 as the scaling center, the actual adaptation position index of the ItemView on the screen is changed, but the actual layout position index of each ItemView remains unchanged. This scaling scheme ensures that the content centered on the two fingers remains unchanged and maintains the continuity of the displayed image. As an example, users can also swipe their fingers up and down on the screen to trigger the phone to update LP and AP, thereby displaying a scrolling grid interface; this will not be elaborated upon here. It should be noted that... Figure 25 This explanation uses the LP and AP encoding method of a low-density view as an example. The encoding method of LP and AP for high-density views is similar and will not be repeated here.

[0178] based on Figure 19 The functional modules shown below, in conjunction with Figures 26 to 30 This section provides a detailed explanation of the specific interactions between various functional modules during the implementation of a grid-based interface design based on spliced ​​images.

[0179] In some embodiments, when the electronic device receives a user's operation on a photo album card, it first initializes the low-density view and the high-density view, and by default displays a 3-level daily view based on the low-density view. The user can trigger the electronic device to display a 5-level daily view based on the low-density view by zooming. Then, if the user continues to decrease the distance between their two fingers, the electronic device displays a 13-level monthly view based on both high and low density views. Subsequently, if the user continues to decrease the distance between their two fingers, the electronic device displays a 31-level yearly view based on both high and low density views.

[0180] For example, Figure 26 The flowchart illustrates the method for switching from a low-density view to a high-density view in a zoomed-out scene.

[0181] like Figure 26 As shown, the method may include the following S1 to S34.

[0182] S1, the touch driver samples the user's touch operation according to the period T.

[0183] The aforementioned cycle, also known as the deployment cycle or reporting cycle, is the cycle for the touch driver to collect user touch operations.

[0184] Taking a touch driver that collects data 240 times per second as an example, the period T = 1 / 240 = 4.16 ms, meaning the touch driver collects touch data once every 4.16 ms. If, during a certain period, the user presses the touchscreen with one or two fingers, the touch driver can determine the touch position of the single or two fingers on the display screen based on voltage changes, etc.

[0185] S2, the touch driver receives the user's click on the gallery application icon and launches the gallery application. The homepage control module displays the album card page.

[0186] For example, when a user wants to view pictures or videos in a gallery app, the user can click on such... Figure 1 The gallery icon 01 is shown in (a) above. The touch driver receives the user's click on the gallery icon 01 and passes the click operation to the application processor. The application processor launches the gallery application corresponding to the gallery icon 01. After launching the gallery application, the gallery application displays as shown below. Figure 1 The card page shown in (b) is an example. The card page provides album cards for the camera, all photos, and videos; therefore, it is also called an album card page. Each album card serves as an entry point to an album.

[0187] It should be noted that the gallery application typically displays album cards by default. If the user clicks on... Figure 1 If the photos, memories, or creations in the bottom bar shown in (b) are used, the homepage control module will switch to other pages.

[0188] S3, the touch driver receives the user's click on a card in the album card page and notifies the gallery application. The gallery application then notifies the grid control module to load the low-density view and the high-density view.

[0189] S4, the grid control module loads the low-density view and the high-density view. The low-density view and the high-density view are initialized. Low-density view initialization includes: initializing the number of columns in the two daily view levels and initializing the size of the low-density view. High-density view initialization includes: initializing the number of columns in the monthly view and the yearly view, and initializing the size of the high-density view.

[0190] Low-density Views inherit from the RecyclerView control, while high-density Views inherit from Android-based View controls. RecyclerView and Android-based View controls are two different types of View controls. RecyclerView is a special type of View control, composed of multiple ItemViews arranged in a grid. Each ItemView displays an image view, and each ItemView supports user touch input on individual ItemViews, making it suitable for day views. Android-based View controls are ordinary View controls, which can be viewed as a blank canvas and do not have the concept of ItemViews. The individual grids in Android View controls are not containers and do not support user touch input on individual ItemViews, but they offer advantages in performance and memory efficiency, making them suitable for month and year views.

[0191] The initialized low-density view and high-density view can have the same or different sizes. Both low-density and high-density views support scaling at different levels. Figures 20 to 23 As shown, the album window is 1300*3470 pixels wide, the low-density view is 2170*6076 pixels wide, and the high-density view is 3100*7200 pixels wide. The low-density view consists of multiple ItemViews in a grid format, while the high-density view is a blank canvas and does not have the concept of ItemView.

[0192] It should be noted that the embodiments in this application are illustrated using a low-density view comprising a daily view with two levels, and a high-density view comprising a monthly view with one level and a yearly view with one level. In actual implementation, the daily view may include one level, three levels, or more levels, and the monthly and yearly views may include two or more levels. Furthermore, the number of columns in each level of the low-density and high-density views can be adjusted according to the screen width, window width, etc.

[0193] S5, the grid main control module notifies the low-density View to initialize the first gear (gear 3) day view.

[0194] S4 above initializes the number of columns in the low-density view at the first level (level 3), for example, setting it to 3 columns. The difference from S4 is that S5 also requires specifying the specific location of the day view drawing area in the low-density view, such as which row and column of the grid the day view drawing area is located in.

[0195] S6, the low-density view triggers the low-density data loader to load data according to the default settings (AP range 1).

[0196] Accordingly, the low-density data loader loads the original image for AP range 1.

[0197] As an example, the default AP range for electronic devices is [1, x1]. AP = 1 represents the thumbnail of the first picture or video in the album, and AP = x1 represents the thumbnail of the x1th picture or video in the album. Here, x1 is a preset value, which is the total number of grids in the grid interface when x1 is greater than or equal to 3.

[0198] As another example, the default AP range 1 for electronic devices is [x1, y1]. AP = y1 represents the thumbnail of the last picture or video in the album, and AP = x1 represents the thumbnail of the x1th picture or video in the album. Here, y1 - x1 is a preset value, representing the total number of grid cells in the grid interface when y1 - x1 is greater than or equal to level 3.

[0199] S7, the low-density view retrieves the original image of the already loaded AP range 1 from the low-density data loader.

[0200] S8, the low-density view draws the first gear (3 gears) of the day view in the day view drawing area based on the image of AP range 1.

[0201] For example, such as Figure 1 As shown in (b), the user can click on the "Camera" card 02. In response to the user's click on the "Camera" card 02, the grid control module loads the low-density view and the high-density view, and initializes the daytime view drawing area of ​​the low-density view in the first position (level 3). Then, the low-density view can call the ondraw() interface to draw the image of AP range 1 in the initialized daytime view drawing area, obtaining the grid interface, so that the phone can display as shown... Figure 1 The album page shown in (c) is shown in the image.

[0202] S9, the grid main control module notifies the high-density view to trigger the loading of stitched images according to the default settings (AP range 1).

[0203] When displaying the first daily view of the album using the default settings, if the user performs continuous zooming operations based on the first daily view, the zooming will inevitably be centered on a specific access point (AP) within the first daily view. This allows the prediction of the AP range of images that might be used when drawing the monthly and yearly views. Based on this, the grid control module can instruct the high-density view to preload the stitched images (such as stitched image 1) for drawing the monthly view and the stitched images (such as stitched image 2) for the yearly view, according to the AP range of the first daily view (i.e., AP range 1).

[0204] For a stitched image of the moon view, the high-density bitmap loader can execute the following steps S10-S12.

[0205] For stitched images of year views, the high-density bitmap loader can execute the following steps S13-S15.

[0206] The execution order of S10-S12 and S13-S15 in this embodiment is not limited; they can be executed simultaneously or sequentially.

[0207] S10, the high-density bitmap loader determines the AP range 2 based on AP range 1.

[0208] AP range 1 includes the APs from the first cell to the last cell in the first frame of the day view. AP range 2 is the AP range of the images that may be used when drawing the moon view, calculated based on AP range 1.

[0209] For example, the first-level daily view consists of 3 columns * 8 rows, and the monthly view consists of 13 columns * 30 rows.

[0210] When shifting from 3rd gear to 13th gear, the following relationship is satisfied:

[0211] canvasSpanCount=targetSpanCount*2-1=13*2-1=25;

[0212]

[0213] That is, the stitched image 1 used to draw the lunar view consists of a 25-column * 58-row grid.

[0214] The high-density view can also calculate the AP value of the center cell based on the AP values ​​of the first and last cells in the first (3rd) day view. Then, this AP value is set as the AP value of the center cell of the stitched image 1. Then, according to the rule that the AP values ​​of adjacent cells in row 1 differ by 1, and the AP value of the i-th cell in row j differs from the AP value of the i-th cell in row j+1 by targetSpanCount, the AP value of each cell in the stitched image 1 is calculated, i.e., the AP range is 2.

[0215] S11, the high-density bitmap loader requests the original image of AP range 2 from the high-density data loader.

[0216] After the high-density data loader completes loading the original image for AP range 2, the high-density data loader returns the original image for AP range 2 to the high-density bitmap loader.

[0217] S12, the high-density bitmap loader stitches the original images in AP range 2 to obtain stitched image 1.

[0218] Referring to the description of S10 in the above embodiment, the stitched image 1 includes a 25-column * 58-row grid. After obtaining the original image corresponding to each grid through S11, the high-density bitmap loader can draw these original images onto each grid according to the AP value of the original images, thereby obtaining the stitched image 1.

[0219] S13, the high-density bitmap loader determines the AP range 3 based on AP range 1.

[0220] AP range 1 includes the APs from the first cell to the last cell in the first frame of the day view. AP range 3 is the AP range of the images that may be used when drawing the year view, calculated based on AP range 1.

[0221] For example, the first-level daily view consists of 3 columns * 8 rows, and the yearly view consists of 31 columns * 72 rows.

[0222] When shifting from 3rd gear to 31st gear, the following relationship is satisfied:

[0223] canvasSpanCount=targetSpanCount*2-1=31*2-1=61;

[0224]

[0225] That is, the stitched image 2 used to draw the year view consists of a 61-column * 143-row grid.

[0226] The high-density view can also calculate the AP value of the center cell based on the AP values ​​of the first and last cells in the first (3rd) day view. Then, this AP value is set as the AP value of the center cell of the stitched image 2. Then, according to the rule that the AP values ​​of adjacent cells in row 1 differ by 1, and the AP value of the i-th cell in row j differs from the AP value of the i-th cell in row (j+1) by targetSpanCount, the AP value of each cell in the stitched image 2 is calculated, i.e., the AP range is 3.

[0227] S14, the high-density bitmap loader requests the original image of AP range 3 from the high-density data loader.

[0228] After the high-density data loader completes loading the original image for AP range 3, the high-density data loader returns the original image for AP range 3 to the high-density bitmap loader.

[0229] S15, the high-density bitmap loader stitches the original images in AP range 3 to obtain stitched image 2.

[0230] Referring to the description of S13 in the above embodiment, the stitched image 2 includes a grid of 61 columns and 143 rows. After obtaining the original image corresponding to each grid through S14, the high-density bitmap loader can draw these original images to each grid according to the AP value of the original images, thereby obtaining the stitched image 2.

[0231] S16, when displaying the daytime view of the first gear, the touch driver receives the user's two-finger press operation and notifies the low-density view.

[0232] S17, Low-density View triggers original image loading based on AP range 1 of the image on the current screen.

[0233] Accordingly, the low-density data loader determines AP range 4 based on AP range 1 and loads the original image of AP range 4.

[0234] When displaying the first level (level 3) daytime view, if the touch driver receives a two-finger press operation from the user, it indicates that the user may need to switch from level 3 to level 5. The low-density view can trigger the loading of the original image corresponding to the second level (level 5) based on the AP range 1 of the image currently displayed on the screen, thus facilitating the direct reading of the already loaded original image from the low-density data loader when switching from level 3 to level 5.

[0235] The low-density view can determine the AP range 4 based on the AP of the first cell (which can be represented as startAP1) and the AP of the last cell (which can be represented as endAP1) when displaying the day view of the first gear (gear 3).

[0236] As an example, the AP of the first image in AP range 4 is equal to startAP1 minus a value, and the AP of the last image in AP range 4 is equal to endAP1 plus a value. This value represents the number of grids included in the drawing area when displaying the second (5th) level day view on the screen.

[0237] Taking a day view with startAP = 100, endAP = 123, and the second tier (tier 5) comprising 5 columns * 13 rows = 65 grids as an example, the AP of the first image in AP range 4 is 100 - 65 = 35, and the AP of the last image in AP range 4 is 123 + 65 = 188. It should be noted that if the AP of the first image in AP range 4 calculated using the above algorithm is negative, it should be corrected to 0 or 1, where 0 or 1 represents the first image in the image library.

[0238] S18, after two fingers are pressed, the touch driver receives the user's operation of reducing the distance between the two fingers and notifies the low-density view.

[0239] S19, the low-density view determines whether the condition for switching from the first gear (gear 3) to the second gear (gear 5) is met based on the distance between two fingers (referred to as the daytime view switching condition).

[0240] In some embodiments, the first gear (gear 3) corresponds to a first scaling factor, and the second gear (gear 5) corresponds to a second scaling factor, where the first scaling factor is greater than the second scaling factor. The average of the first and second scaling factors is taken as the gear switching threshold. A scaling factor Si is calculated based on the user's action of reducing the distance between their two fingers. The scaling factor Si is compared to the gear switching threshold. If Si is less than or equal to the gear switching threshold, then the condition of switching from the first gear (gear 3) to the second gear (gear 5) is met; if the scaling factor Si is greater than the gear switching threshold, then the condition of switching from the first gear (gear 3) to the second gear (gear 5) is not met.

[0241] For example, the touch driver can first receive the user's two-finger press operation, and then receive the user's operation of reducing the distance between the two fingers.

[0242] When the touch driver receives a two-finger press operation from the user, it can transmit the coordinates (x11, y11) and (x12, y12) of the two-finger press to the zoom gesture detector. Figure 26(Not shown). The zoom gesture detector calculates the coordinates of the two-finger center (mBeginFocusx, mBeginFocusy) ​​and the distance d1 between the two fingers based on the coordinates (x11, y11) and (x12, y12).

[0243] The coordinates of the two-finger center (mBeginFocusx, mBeginFocusy) ​​can be obtained through the following relationship (1):

[0244]

[0245] The distance d1 between the two fingers can be obtained by the following relationship (2):

[0246]

[0247] Taking the coordinates (x11, y11) = (200, 200) and (x12, y12) = (800, 1000) when two fingers are pressed down as an example, after the zoom gesture detector calculates the two-finger spacing d1 = 1000 pixels, it can be determined that the two-finger spacing d1 = 1000 is greater than the threshold D = 540 pixels, thus determining that the operation is valid, and recording the two-finger center coordinates (500, 600) and the two-finger spacing d1 = 1000 pixels.

[0248] Then, the touch driver receives the user's action to reduce the distance between two fingers and passes the coordinates (xi1, yi1) and (xi2, yi2) after the two-finger movement to the scaling gesture detector. i is an integer greater than or equal to 2. i represents the i-th time a two-finger action is detected. The scaling gesture detector calculates the distance between the two fingers, di, based on the coordinates (xi1, yi1) and (xi2, yi2).

[0249] The distance between the two fingers, di, can be obtained through the following relationship (3):

[0250]

[0251] Taking the coordinates (xi1, yi1) = (100, 100) and (xi2, yi2) = (460, 580) after a two-finger movement operation as an example, after the scaling gesture detector calculates the two-finger spacing d1 = 600 pixels, it can be determined that the two-finger spacing di = 600 is greater than the threshold D = 540 pixels, thus the operation is deemed valid, and the scaling factor Si is calculated based on the two-finger spacing di and d1.

[0252] The scaling factor Si can be obtained through the following relationship (4):

[0253]

[0254] scale1 is the scaling factor when two fingers are pressed. di is the two-finger spacing corresponding to the i-th scaling operation, and d1 is the two-finger spacing corresponding to the two-finger pressing operation. It should be understood that if the user reduces the distance between the two fingers, then di < d1; if the user increases the distance between the two fingers, then di > d1; if the two fingers of the user do not move, the scaling factor is equal to 1, which is determined to be invalid, and there is no need to determine whether to set the two-finger center as the scaling center. In one period T, the faster the two fingers move, the larger the difference between di and d1, and the faster the gear shift is performed.

[0255] As an example, the electronic device may pre-configure a plurality of gears, and a gear shift critical point is set between any two adjacent gears. Two adjacent gears refer to that no other gears are arranged between the two gears. If the scaling factor of the small gear is represented by a1, the scaling factor of the large gear is represented by a2, and the gear shift critical point is represented by a3, the gear shift critical point a3 can be calculated through the following relational expression (5):

[0256]

[0257] Taking that the scaling factor of the low-density View at gear 3 is 1.0 and the scaling factor at gear 5 is 0.6 as an example, as shown in Table 1, it can be calculated according to the above relational expression (5) that the gear shift critical point between gear 3 and gear 5 is 0.8.

[0258] Table 1

[0259] 3rd gear 1.0 Gear shifting threshold 0.8 5 gears 0.6

[0260] After the electronic device obtains the scaling factor Si based on the relational expression (4), it may compare the magnitude of the scaling factor Si with the gear shift critical point 0.8. If Si ≤ 0.8, it may be determined that the day view switching condition is satisfied, and the following S13 is executed. If Si > 0.8, it may be determined that the day view switching condition is not satisfied, and there is no need to switch from the day view of the first gear (gear 3) to the day view of the second gear (gear 5).

[0261] S20: The low-density View determines an AP range 5 based on the scaling center.

[0262] The scaling center is exactly the two-finger center calculated in S19, or a position obtained by performing left-right correction on the two-finger center. It should be noted that the two-finger center remains unchanged from the moment the user presses the two fingers to the moment the user lifts the fingers.

[0263] For the specific implementation manner of determining the AP range based on the scaling center, reference may be made to the following embodiments for Figure 30 the description.

[0264] S21, the low-density view obtains the original image of AP range 5 from the original image of AP range 4.

[0265] S22, the low-density View, based on an image within AP range 5, draws a second-level (level 5) daytime view within the daytime view drawing area. Then, the low-density View displays the second-level (level 5) daytime view, allowing the user to see the displayed image. Figure 1 The daytime view of the second gear (5th gear) shown in (e) is shown in the image.

[0266] After the electronic device displays the second (5th) level of the daytime view, the user can continue to reduce the distance between the two fingers without leaving the screen, thereby the electronic device performs the following S23.

[0267] S23, after displaying the daytime view at the second level (level 5), the touch driver receives the user's operation of continuing to reduce the two-finger spacing and notifies the low-density view.

[0268] S24, the low-density view determines whether the condition for switching from the second gear (5th gear) to the third gear (13th gear) is met (called the moon view switching condition).

[0269] Taking a low-level day view with 3 columns, a high-level day view with 5 columns, and a moon view with 13 columns as an example, with the scaling factor of the day view being 0.6 at level 5. Since the day view is based on a low-density view and the moon view is based on a high-density view, and the number of columns in the low-density view and the high-density view are different, when switching from the day view to the moon view, the moon view can be converted to a low-density view to obtain the scaling factor 3 / 13 = 0.23 corresponding to the moon view. According to the above relationship (5), the critical point for switching between levels 5 and 13 can be calculated as (0.6 + 0.23) / 2 = 0.42. If the scaling factor corresponding to the operation of reducing the two-finger spacing is less than or equal to 0.42, then the moon view switching condition is met, and the following S25 can be executed. If the scaling factor corresponding to the operation of reducing the two-finger spacing is greater than 0.42, then the moon view switching condition is not met, and the electronic device can continue to detect the user's operation of reducing the two-finger spacing and determine whether the newly detected operation meets the moon view switching condition. It should be noted that the scaling factor corresponding to the operation of reducing the distance between two fingers can be calculated by the above (1)-(4), which will not be repeated here.

[0270] S25, the low-density view notifies the high-density view to switch to the moon view.

[0271] Accordingly, the high-density view determines the AP range 6 based on the zoom center.

[0272] As an example, a high-density view can determine the drawing area of ​​the grid interface to be drawn within the high-density view based on the zoom center. Then, based on the LP value and AP value of the selected grid at the zoom center, the AP value of the first grid and the AP value of the last grid in that drawing area are determined, with one AP value corresponding to one image to be stitched. That is, the AP range from the AP value of the first grid to the AP value of the last grid is 6.

[0273] For a specific implementation method of determining the AP range based on the zoom center, please refer to the following embodiment. Figure 30 The description.

[0274] S26, the high-density view requests the image corresponding to AP range 6 from the high-density bitmap loader.

[0275] S27, the high-density bitmap loader crops a large image corresponding to AP range 6 from the stitched image 1.

[0276] Accordingly, the high-density bitmap loader returns a large image corresponding to AP range 6 to the high-density view.

[0277] In the large image corresponding to AP range 6, the AP values ​​of each cell satisfy the following rules: the AP values ​​of adjacent cells in row 1 differ by 1, and the AP value of the image in cell i in row j differs from the AP value of the image in cell i in row (j+1) by targetSpanCount = 13, where i and j are positive integers.

[0278] by Figure 12 Taking the stitched image shown as an example, the cropping area enclosed by line 1 can be used to draw, as shown in the image. Figure 8 The 5-column day view shown, with the cropped area enclosed by line type 2, can be used to draw, as... Figure 9 The 5-column day view shown, with the cropped area enclosed by line type 3, can be used to draw, for example... Figure 7 The image shows a 5-column daily view. It should be noted that... Figure 12 The stitched image shown only includes a 9-column * 23-row grid. In reality, stitched image 1 contains a much larger number of grids. For example, according to the calculation result of S11, stitched image 1 includes a 25-column * 58-row grid. Due to the limitations of the attached drawing size, this embodiment of the application will not show a schematic diagram of stitched image 1.

[0279] S28, the high-density view draws the moon view at the third level (level 13) based on the large map corresponding to AP range 6. Then, the high-density view displays the moon view, allowing the user to see the display as shown. Figure 1 The lunar view shown in (f) is shown in the image.

[0280] After displaying the moon view, as described in S29 to S35 below, the user can continue to reduce the distance between the two fingers without leaving the screen, thereby the electronic device performs S29 below.

[0281] S29, after displaying the third-level moon view, the touch driver receives the user's operation of continuing to reduce the two-finger spacing and notifies the high-density view.

[0282] S30, the high-density view determines whether the condition for switching from the third gear (13th gear) to the fourth gear (31st gear) is met (called the year view switching condition).

[0283] The implementation of the "year view switching condition" is similar to that of the "day view switching condition" and "month view switching condition". Please refer to the specific description of the above embodiments, which will not be repeated here.

[0284] If the year view switching conditions are met, then S31 can be executed as follows. If the year view switching conditions are not met, then user operations can continue to be detected, and it can be determined whether the newly detected operations meet the year view switching conditions.

[0285] S31, High-density View determines AP range 7 based on zoom center.

[0286] As an example, a high-density view can determine the drawing area of ​​the grid interface to be drawn within the high-density view based on the zoom center. Then, based on the LP value and AP value of the selected grid at the zoom center, the AP value of the first grid and the AP value of the last grid in that drawing area are determined, with one AP value corresponding to one image to be stitched. That is, the AP range from the AP value of the first grid to the AP value of the last grid is 7.

[0287] For a specific implementation method of determining the AP range based on the zoom center, please refer to the following embodiment. Figure 30 The description.

[0288] S32, the high-density view requests the image corresponding to AP range 7 from the high-density bitmap loader.

[0289] S33, the high-density bitmap loader crops a large image corresponding to AP range 7 from the stitched image 2.

[0290] Accordingly, the high-density bitmap loader returns a large image corresponding to AP range 7 to the high-density view.

[0291] In the large image corresponding to AP range 7, the AP values ​​of each cell satisfy the following rules: the AP values ​​of adjacent cells in row 1 differ by 1, and the AP value of the image in cell i in row j differs from the AP value of the image in cell i in row (j+1) by targetSpanCount = 31, where i and j are positive integers.

[0292] For the specific implementation of S33, please refer to the description of S27 above, which will not be repeated here.

[0293] S34, the high-density view draws the year view for the fourth tier (tier 31) based on the large image corresponding to AP range 7. Then, the high-density view displays the year view, allowing the user to see the display as shown. Figure 2 The year view shown in (a) is shown in the image.

[0294] In the above scheme, when the photo album is opened, the electronic device first displays the daily view with fewer grid columns, followed by the monthly and yearly views with more grid columns, which better suits user habits. When displaying the daily view, since the number of images to be displayed is small, the number of images the GPU needs to render is also small. Using an image stitching scheme would result in little or no reduction in overall processing time. Therefore, a grid interface rendering scheme based on low-density views and non-stitched images is adopted. When displaying the monthly and yearly views, the number of images the GPU needs to render is large. A grid interface rendering scheme based on high-density stitched images is adopted. By stitching multiple images, the number of image data and the number of times the GPU renders images can be reduced by an order of magnitude, improving the GPU's processing speed and making the photo album interface display smoother.

[0295] It should be noted that when displaying the daily view, users require grid-based touch interaction. The custom RecyclerView control's ItemViews support responding to user touch operations on individual ItemViews; therefore, a custom RecyclerView was used as a low-density view. When displaying the monthly or yearly view, users do not require grid-based touch interaction. Android-based view controls offer advantages in performance and memory efficiency; therefore, Android-based view controls were used as high-density views.

[0296] The above embodiment illustrates how, when an electronic device receives a user's operation on a photo album card, the device defaults to displaying a low-density view with 3 levels of daily view, loading a stitched image 1 corresponding to the monthly view and a stitched image 2 corresponding to the yearly view. Then, without lifting two fingers from the screen, continuous zooming triggers the electronic device to sequentially switch between 5 levels of daily view, 13 levels of monthly view, and 31 levels of yearly view. In actual implementation, the user might lift their hand at any level and then perform a swipe operation; or, the user might lift their hand at any level and then press down with two fingers again to zoom; or, the user might change the zoom trend, such as from decreasing the distance between two fingers to increasing the distance between two fingers, or vice versa. These operations may all trigger the electronic device to update the AP value of the grid interface. Referring to the description of the above embodiment, stitched image 1 and stitched image 2 are calculated based on the default AP range 1. After the electronic device updates the AP value of the grid interface, the AP range of stitched image 1 and stitched image 2 may not be applicable to the current monthly and yearly view rendering. Based on this, this application also proposes a scheme to reload the stitched image according to the current AP value of the grid interface after updating the AP value of the grid interface.

[0297] For example, Figure 27 This diagram illustrates a method for reloading stitched images after scrolling through a photo album page based on a high-density view.

[0298] like Figure 27 As shown, the method may include the following steps S35 to S47.

[0299] In the S35 scenario, when displaying a month or year view based on the fifth position, the touch driver receives an up or down swipe operation from the user on the display screen and notifies the high-density view. The high-density view then triggers rendering.

[0300] The fifth gear can be either the third gear (13 gears) or the fourth gear (31 gears).

[0301] S36, High-density View redraws the month view or year view based on the sliding distance.

[0302] When a user swipes their finger up or down on the screen, from the user's perspective, the images in the grid interface should also swipe up or down, thus triggering an AP refresh in the grid interface.

[0303] When a high-density view supports vertical scrolling, if the user swipes their finger up or down on the screen, the high-density view moves vertically relative to the display area. The positions of each grid cell within the high-density view also move vertically relative to the display area. In this case, the high-density view can calculate the AP (Area of ​​Power) for each grid cell in the last grid cell after scrolling stops. In the last grid cell, the AP range from the first grid cell to the last grid cell is represented as AP range-a.

[0304] As an example, a high-density view can update the AP value of each cell according to the formula: AP + s * g. Here, s represents the final number of rows the high-density view moves, and g represents the current high-density view's level (i.e., the number of cells / columns in the drawing area).

[0305] To better understand how to trigger the drawing of a high-density View in a scrolling scenario Figure 28 This illustration shows a scenario where a user swipes a single finger upwards relative to the screen (also known as in the first direction), causing the bottom of a View control to slide upwards into the screen, and the last row of the View control is not the last row of the photo album. It should be noted that the size of the accompanying image is limited. Figure 28 This explanation uses level 3 as an example. In reality, high-density views have 13 or 31 columns.

[0306] like Figure 28 As shown in (a), the phone displays a first grid interface based on a View control. The AP of the first image in the display area is 160, and the AP of the last image in the display area is 180. The user presses and holds the area corresponding to the actual adaptation position index 170 with a single finger, and continues to slide upwards for a distance of 300 pixels (also known as the second distance). At this moment, the distance from the bottom edge of the display to the bottom edge of the recycle bin view is the first distance. In the N+1th reporting cycle, the phone detects an upward slide distance of 300 pixels, i.e., dy = -300 pixels. In response to the user's single-finger upward slide operation (also known as the first operation), as follows... Figure 28 As shown in (b), the phone moves the View control upwards by 300 pixels. At this moment, the Y-axis coordinate of the top of the View control is updated to y1 = -1900 - 300 = -2200 pixels, and the Y-axis coordinate of the bottom of the View control is updated to y2 = 2500 - 300 = 2200 pixels.

[0307] Then, the phone performs the following three judgment steps in sequence.

[0308] First judgment step:

[0309] The mobile phone retrieves the actual layout position index of the ItemView within the View control. For example... Figure 28 As shown in (b) in the drawing, the first actual layout position index value located at the first row and first column of the View control is LP1=246, and the last actual layout position index value located at the last row and last column of the View control is LP2=300. Herein, LP1 and LP2 can be calculated according to the actual AP value of the View control. Then, since the current scenario is a single-finger upward sliding scenario, and the View control slides upward as the single finger moves, the mobile phone only needs to compare the magnitudes of LP2=300 and LPmax=835. Since the actual LP2 is less than the actual LPmax, the mobile phone can determine that the last row of the View control is not the last row of the photo album, and the View control has not yet carried the last image of the photo album. If the user continues to slide the finger upward, the View control can still update the image corresponding to each ItemView, so that the user can see other images with larger actual LP values that have not been displayed in the photo album. Therefore, the mobile phone does not need to perform the first position correction on the View control, and D1=0.

[0310] The second determining step:

[0311] The mobile phone compares the Y-axis coordinate y2=2200 pixels at the bottom of the View control with the Y-axis coordinate Y2=2400 pixels at the bottom of the screen. Since y2<Y2, the bottom of the View control has been slid into the screen. In order to enable the View control to carry more images with larger actual LP values, the mobile phone needs to move the View control downward (also referred to as the second direction).

[0312] For example, the mobile phone can calculate the number of rows to move downward according to the relational expression:

[0313]

[0314] Correspondingly, the distance that the View control moves downward (also referred to as the third distance) is: 400 pixels * 1 row = 400 pixels, that is, D2=400.

[0315] As shown Figure 28 in (c) in the drawing, after the View control moves downward by a distance of 400 pixels, the Y-axis coordinate at the bottom of the View control is updated to y2=2200+400=2600 pixels. In addition, the mobile phone updates the actual adaptation position index value of each ItemView according to the relational expression of actual AP + 1 row * 3 columns, that is, the AP value corresponding to each item view is added with a numerical value. For example, the actual AP value of the first row of the View control is updated from the values as shown Figure 28 in (b) in the drawing, which are 148, 149 and 150, to 151, 152 and 153 respectively as shown Figure 28 in (c) in the drawing; the actual AP value of the second row of the View control is updated from the values as shown Figure 28 151, 152, and 153 shown in (b) in the figure are respectively updated to as shown in Figure 28 154, 155, and 156 shown in (c) in the figure; ...; the actual AP values of the last row of the View control are updated from as shown in Figure 28 178, 179, and 180 shown in (b) in the figure to as shown in Figure 28 181, 182, and 183 shown in (c) in the figure.

[0316] It can be understood that in the second judgment step, the mobile phone first moves the View control down by 1 row, then adds 3 to the AP values corresponding to the ItemViews from the 2nd column to the 5th column of the View control respectively, which is equivalent to not moving the ItemViews, so that the positions of each image on the screen remain unchanged from the user's visual perception.

[0317] Third judgment step:

[0318] The mobile phone acquires the actual layout position index value of the last ItemView of the current View control again. In the second judgment step, the View control is moved down by one row, so the actual layout position index value of each ItemView in the View control is increased by 5 (the number of columns of the View control). As shown in Figure 28 the (d) in the figure, the first actual layout position index value of the current View control is updated to LP1=246+5=251, and the last actual layout position index value is updated to LP2=300+5=305. Then, the magnitudes of LP2=305 and LPmax=835 are judged. Since LP2<LPmax, the last row of the View control is not the last row of the album, and the View control has not yet carried the last image of the album. If the user continues to slide the finger upward, the View control can still update the images corresponding to each ItemView, so that the user can see other images with larger actual LP values that have not been displayed in the album. Therefore, the mobile phone does not need to perform a second position correction on the View control, that is, D3=0.

[0319] After obtaining dy=-300, D1=0, and D3=0 through the above steps, the mobile phone can obtain the final movement distance dy+D1+D3=-300, and refresh the screen display content. In the refreshed display area, the AP of the first picture is updated to 163, and the AP of the last picture is updated to 183, that is, the AP range x is [163, 183]. From the user's perspective, the user sees that the album interface as shown in Figure 28 (a) in the figure is directly updated to as shown in Figure 28 (e) in the figure, that is, the album interface slides upward following the user's finger by 300 pixels, and a new row of images is added at the bottom of the screen.

[0320] S37, the high-density view triggers the loading of stitched images based on the AP range range-a of the images on the current screen.

[0321] S38, the high-density bitmap loader determines the AP range range-b based on the AP range range-a.

[0322] The AP range range-a mentioned above includes the APs from the first cell to the last cell in the redrawn monthly or yearly view. The AP range range-b mentioned above is the AP range calculated based on the AP range range-a for drawing images that may be used in the yearly view when switching from the monthly view, or the AP range for drawing images that may be used in the monthly view when switching from the yearly view.

[0323] S39, the high-density bitmap loader requests the original image of the AP range range-b from the high-density data loader. The high-density data loader loads the original image of the AP range range-b. Afterwards, the high-density data loader returns the original image of the AP range range-b to the high-density bitmap loader.

[0324] S40, the high-density bitmap loader stitches together the original images within the AP range-b to obtain stitched image 3.

[0325] For the specific implementation of S38-S40, please refer to the descriptions of S10-S12 and S13-S15 above, which will not be repeated here.

[0326] S41, after the month view or year view is redrawn, the touch driver receives the user's two-finger press and decrease / increase the two-finger spacing operation, and notifies the high-density view.

[0327] S42, the high-density view determines whether the condition for switching from the fifth gear to the sixth gear is met.

[0328] Specifically, when a user reduces the distance between two fingers, the fifth position can be the third position and the sixth position can be the fourth position; or, when a user increases the distance between two fingers, the fifth position can be the fourth position and the sixth position can be the third position.

[0329] The implementation of the "condition for switching from the fifth gear to the sixth gear" is similar to that of the "condition for switching the day view" and the "condition for switching the month view". Please refer to the specific description in the above embodiments, which will not be repeated here.

[0330] If the condition for switching from the fifth gear to the sixth gear is met, then S43 can be executed as follows. If the condition for switching from the fifth gear to the sixth gear is not met, then the user operation can continue to be detected, and it can be determined whether the newly detected operation meets the condition for switching from the fifth gear to the sixth gear.

[0331] S43, High-density View determines the AP range range-c based on the zoom center.

[0332] As an example, a high-density view can determine the drawing area of ​​the grid interface to be drawn within the high-density view based on the zoom center. Then, based on the LP value and AP value of the selected grid at the zoom center, the AP value of the first grid and the AP value of the last grid in that drawing area are determined, with one AP value corresponding to one image to be stitched. That is, the AP range from the AP value of the first grid to the AP value of the last grid is range-c.

[0333] For a specific implementation method of determining the AP range based on the zoom center, please refer to the following embodiment. Figure 30 The description.

[0334] S44, the high-density view requests the image corresponding to the AP range range-c from the high-density bitmap loader.

[0335] S45, the high-density bitmap loader crops a large image corresponding to the AP range range-c from the stitched image 3.

[0336] S46, the high-density bitmap loader returns a large image corresponding to the AP range range-c to the high-density view.

[0337] In the large image corresponding to the AP range range-c, the AP values ​​of each cell satisfy the following rules: the AP values ​​of adjacent cells in row 1 differ by 1, and the AP value of the image in the i-th cell of row j differs from the AP value of the image in the i-th cell of row (j+1) by targetSpanCount, where i and j are positive integers.

[0338] S47, High-density View draws the sixth-level monthly or yearly view based on the large image corresponding to the AP range range-c.

[0339] For the specific implementation of S41-S47, please refer to the descriptions of S23-S28 and S29-S34 above, which will not be repeated here.

[0340] In the above solution, when the electronic device displays a monthly or yearly view, the user can swipe their finger up or down on the screen to trigger the device to update the AP range of the current image on the screen. It can be understood that after the AP range of the current image on the screen is updated, the electronic device can reload the stitched image based on the AP range of the current image on the screen. This allows for quick cropping of a larger image from the reloaded stitched image when the user performs zoom operations, and then rendering this larger image into the View control. Since this larger image includes multiple original images, this significantly reduces the amount of image data and the number of times the GPU needs to render images, improving GPU processing speed and making the album interface display smoother.

[0341] For example, Figure 29 This diagram illustrates a method for drawing a month view based on a stitched image prepared when displaying a year view, as the user reduces the distance between two fingers.

[0342] like Figure 29 As shown, the method may include the following steps S48 to S58.

[0343] S48, when displaying the year view of the fourth level (31 levels), the high-density view triggers the loading of the stitched image based on the AP range range-d of the image on the current screen.

[0344] S49, the high-density bitmap loader determines the AP range range-e based on the AP range range-d of the image on the current screen.

[0345] The AP range range-d mentioned above includes the APs from the first cell to the last cell in the year view. The AP range range-e mentioned above is the AP range calculated based on the AP range range-d, which is the range of images that may be used to draw the month view when switching from the year view to the month view.

[0346] S50, the high-density bitmap loader requests the original image within the AP range (range-e) from the high-density data loader. The high-density data loader loads the original image within the AP range (range-e). Afterwards, the high-density data loader returns the original image within the AP range (range-e) to the high-density bitmap loader.

[0347] S51, the high-density bitmap loader stitches together the original images within the AP range-e to obtain stitched image 4.

[0348] For the specific implementation of S48-S51, please refer to S10-S12, S13-S15 above, and the implementation of S48-S51. Figures 16 to 18 The description will not be repeated here.

[0349] S52, when displaying the year view, the touch driver receives the user's operation of pressing down with two fingers and increasing the distance between the two fingers, and notifies the high-density view.

[0350] S53, the high-density view determines whether the condition for switching from the fourth gear (31 gears) to the third gear (13 gears) is met.

[0351] The implementation of the "condition for switching from the fourth gear to the third gear" is similar to that of the "condition for switching the day view" and the "condition for switching the month view". Please refer to the specific description in the above embodiments, which will not be repeated here.

[0352] If the condition for switching from the fourth gear to the third gear is met, then S54 can be executed. If the condition for switching from the fourth gear to the third gear is not met, then the user operation can continue to be detected, and it can be determined whether the newly detected operation meets the condition for switching from the fourth gear to the third gear.

[0353] S54, High-density View determines the AP range range-f based on the zoom center.

[0354] As an example, a high-density view can determine the drawing area of ​​the grid interface to be drawn within the high-density view based on the zoom center. Then, based on the LP value and AP value of the selected grid at the zoom center, the AP value of the first grid and the AP value of the last grid in that drawing area are determined, with one AP value corresponding to one image to be stitched. That is, the AP range from the AP value of the first grid to the AP value of the last grid is range-f.

[0355] For a specific implementation method of determining the AP range based on the zoom center, please refer to the following embodiment. Figure 30 The description.

[0356] S55, the high-density view requests the image corresponding to the AP range range-f from the high-density bitmap loader.

[0357] S56, the high-density bitmap loader crops a large image corresponding to the AP range range-f from the stitched image 4.

[0358] S57, the high-density bitmap loader returns a large image corresponding to the AP range range-f to the high-density view.

[0359] In the large image corresponding to the AP range range-f, the AP values ​​of each cell satisfy the following rules: the AP values ​​of adjacent cells in row 1 differ by 1, and the AP value of the image in the i-th cell of row j differs from the AP value of the image in the i-th cell of row (j+1) by targetSpanCount, where i and j are positive integers.

[0360] S58, the high-density view draws the third-level moon view based on the large image corresponding to the AP range-f.

[0361] For the specific implementation of S52-S58, please refer to the descriptions of S23-S28 and S29-S34 above, which will not be repeated here.

[0362] It should be noted that the above embodiment is an example of how to draw a month view based on a stitched image prepared when displaying the year view when the user decreases the spacing between two fingers. It can be understood that the methods for drawing a month view based on a stitched image prepared when displaying the day view and for drawing a year view based on a stitched image prepared when displaying the month view when the user increases the spacing between two fingers are similar to the above methods, and will not be described again here.

[0363] In the above solution, when the electronic device displays a yearly view, the area range (AP) from the first to the last cell of the yearly view covers the area range of the monthly view obtained by zooming in on any cell. Therefore, a stitched image can be generated in advance based on this AP range. Thus, when the user presses two fingers on the screen and increases the distance between them, a large image can be quickly cropped from the reloaded stitched image and drawn into the View control. Since this large image includes multiple original images, the amount of image data and the number of times the GPU draws images are reduced by an order of magnitude, improving the GPU's processing speed and making the album interface display smoother.

[0364] The following is combined with Figure 30 This paper explains the specific scheme for determining the AP range based on the scaling center of a high-density view.

[0365] like Figure 30 As shown, the scheme may include the following (1) to (8).

[0366] (1) Determine the scaling center (pointx, mBeginFocusy), the column number of the first column of the next level display area in the high-density view (Index1), and the column number of the last column of the next level display area in the high-density view (Index2).

[0367] Optionally, the high-density view can also determine the row number (Line1) of the first row of the display area of ​​the next tier in the high-density view, and the row number (Line2) of the last row of the display area of ​​the next tier in the high-density view. Based on Index1, Index2, Line1, and Line2, the AP of the first cell and the AP of the last cell in the display area of ​​the next tier can be determined.

[0368] The zoom center (pointx, mBeginFocusy) ​​can be determined as follows: Determine the first distance (screenRectF.left), the second distance (X1), the third distance (X2), and the first width (screenRectF.width). The first distance (screenRectF.left) is the distance from the left edge of the display area of ​​the first grid interface in the high-density view to the left edge of the high-density view; the second distance (X1) is the distance from the left edge of the display area corresponding to the second column number in the high-density view to the left edge of the high-density view; the third distance (X2) is the distance from the right edge of the display area corresponding to the second column number in the high-density view to the left edge of the high-density view; and the first width (screenRectF.width) is the width of the display area of ​​the first grid interface in the high-density view. Based on the first distance (screenRectF.left), the second distance (X1), the third distance (X2), and the first width (screenRectF.width), determine the first coordinate value (centerX), which represents the distance from the zoom center to the left edge of the high-density view. Map the first coordinate value onto the screen to obtain the second coordinate value, pointx. The second coordinate value, pointx, represents the distance from the zoom center to the left edge of the screen. The position corresponding to the second coordinate value, pointx, and the third coordinate value, mBeginFocusy, is determined as the first position. The third coordinate value, mBeginFocusy, represents the distance from the center of the two fingers to the top edge of the screen.

[0369] (2) Determine the LP value (also known as selectLP) of the selected cell at the zoom center, and the AP value (also known as selectAP) of the selected cell at the zoom center.

[0370] The selected grid cell for zoom center refers to the grid cell where the zoom center's coordinates are located.

[0371] The following steps can be performed on each cell in a high-density view:

[0372] (3) Determine the number of columns (LineNum) and LP value of a cell in a high-density view.

[0373] The column number lineNum of the cell to be adjusted can be determined according to the following formula:

[0374] LineNum=LP%maxShowSpan+1.

[0375] Among them, maxShowSpan represents the maximum gear (e.g., 31 gears).

[0376] (4) Determine whether the condition Index1≤LineNum≤Index2 is met.

[0377] If Index1≤LineNum≤Index2 is satisfied, then execute (5).

[0378] If Index1≤LineNum≤Index2 is not satisfied, then execute (8).

[0379] (5) Determine whether LP≤selectLP is satisfied.

[0380] If LP≤selectLP is satisfied, then execute (6).

[0381] If LP≤selectLP is not satisfied, then execute (7).

[0382] (6) Determine the AP value of this cell according to the first formula.

[0383] The first formula can be:

[0384]

[0385] Among them, nextShowSpan represents the number of columns to display in the next level, maxShowSpan represents the maximum level (e.g., 31 levels), selectLP is the LP value of the selected grid cell with zoom in and out, and selectLineNum represents the number of rows selected.

[0386] (7) Determine the AP value of this cell according to the second formula.

[0387] The second formula can be:

[0388]

[0389] (8) Set the AP value to -1.

[0390] In this case, AP = -1 means that the actual adaptation position value is invalid, and at this time, a grid can display a blank image.

[0391] In the above scheme, the AP value of the grid can be refreshed when the cross-level condition is met. Since the AP value of the grid corresponds to the order of the images displayed in the grid in the album, the order of the images displayed on the screen is continuous.

[0392] For example, Figure 31 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0393] like Figure 31 As shown, the electronic device 100 may include a processor 110, internal memory 120, buttons 130, a sensor module 140, a display screen 150, an audio module 160, a speaker 160A, a receiver 160B, a microphone 160C, a headphone jack 160D, etc. The sensor module may include a touch sensor.

[0394] The processor 110 can be used to execute the scrolling display method in the above embodiments. The processor 110 may include one or more processing units, such as a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), an image signal processor (ISP), etc.

[0395] Internal memory 120 can be used to store executable program code, including instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 120. Internal memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application (APP) required for a function, such as a gallery application. The data storage area may store configuration files for each APP, as well as data created during the use of electronic device 100.

[0396] The display screen 150 includes a display panel for displaying various images in a gallery application.

[0397] A touch sensor, also known as a "touch panel," can be located on the display screen 150. The touch sensor and the display screen 150 together form a touchscreen, also called a "touch display." The touch sensor can detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event and provide visual output related to the touch operation through the display screen 150. In some embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 150.

[0398] This application also provides a computer-readable storage medium storing computer instructions. When the computer-readable storage medium is operated on an electronic device, it causes the electronic device to perform the methods shown in the above embodiments. The computer instructions can be stored in the computer-readable storage medium or transferred from one computer-readable storage medium to another. The computer-readable storage medium can be any usable medium that a computer can access, or it can include one or more data storage devices such as servers or data centers that can be integrated with the medium. The usable medium can be a magnetic medium, an optical medium, or a semiconductor medium, etc.

[0399] This application also provides a computer program product, which includes computer program code. When the computer program code is run on an electronic device, it causes the electronic device to perform the methods described in the above embodiments.

[0400] The electronic devices, computer-readable storage media, and computer program products provided in the embodiments of this application are all used to execute the methods provided in the above embodiments.

[0401] In the description of this application, " / " means "or". For example, A / B can mean A or B. In the description of this application, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0402] In the specification and claims of this application, the terms "first" and "second," etc., are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of the objects. For example, "first operation" and "second operation," etc., are used to distinguish different operations, rather than to describe a specific order of operations. In the embodiments of this application, "multiple" refers to two or more.

[0403] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0404] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for displaying a grid interface, characterized in that, The method includes: Receive the user's first operation on the electronic device; In response to the first operation, a first grid interface is displayed, and a first stitched image is obtained based on the adaptation position values ​​of the images in the first grid interface. The first grid interface includes M columns of grids, each row of the first stitched image includes 2N-1 images, the adaptation position values ​​of adjacent grids in each row of the first stitched image differ by 1, and the adaptation position value of the image in the j-th row and i-th grid of the first stitched image differs from the adaptation position value of the image in the (j+1)-th row and i-th grid by N. M, N, i, and j are positive integers, and N is not equal to M. Receive a second operation from the user on the electronic device, wherein the second operation is a zoom operation; In response to the second operation, a first image is cropped from the first stitched image according to the zoom center, and a second grid interface corresponding to the first image is displayed, wherein each row of the first image includes N images, and the second grid interface includes N columns.

2. The method according to claim 1, characterized in that, The first grid interface includes P rows of grids, the second grid interface includes Q rows of grids, each column of the first image includes Q images, and P and Q are positive integers; If N is greater than M, each column in the first stitched image includes Q. 2-(M) P / N) images; If N is less than M, each column in the first stitched image includes M. P / N images.

3. The method according to claim 1, characterized in that, The adaptation position value of the center grid of the first stitched image is equal to the adaptation position value of the center grid of the first grid interface. The adaptation position value of the center grid of the first grid interface is determined based on the adaptation position values ​​of the first grid and the last grid of the first grid interface.

4. The method according to claim 1, characterized in that, The first operation is a tap on an album card, and the second operation is an increase in the distance between two fingers; or, The first operation is a swipe operation on the third grid interface, and the second operation is an operation to increase or decrease the spacing between two fingers. The third grid interface includes M columns, and the adaptation position values ​​of the images in the third grid interface are not exactly the same as the adaptation position values ​​of the images in the first grid interface; or... The first operation is to increase the distance between two fingers, and the second operation is to decrease the distance between two fingers; or, The first operation is to reduce the distance between two fingers, and the second operation is to increase the distance between two fingers.

5. The method according to claim 1, characterized in that, The second grid interface is drawn based on the first view control, and the size of the first view control is greater than or equal to the size of the screen; The step of cropping the first image from the first stitched image based on the scaling center includes: Determine the drawing area of ​​the second grid interface within the first view control; Based on the layout position value and adaptation position value of the selected grid at the zoom center, determine the adaptation position value of the first grid and the adaptation position value of the last grid in the drawing area. One adaptation position value corresponds to one image. Based on the adaptation position values ​​of the first and last grid cells in the drawing area, the first image is cropped from the first stitched image. The adaptation position value of the first grid cell in the first image is equal to the adaptation position value of the first grid cell in the drawing area, and the adaptation position value of the last grid cell in the first image is equal to the adaptation position value of the last grid cell in the drawing area. The first image is drawn onto the drawing area to obtain the second grid interface.

6. The method according to any one of claims 1 to 5, characterized in that, The first operation is a click operation on an album card. Before displaying the first grid interface, the method further includes: Load the first view control and the second view control; Initialize the size of the first view control and the number of columns for each gear corresponding to the first view control; Initialize the size of the second view control and the number of columns for each position corresponding to the second view control; The first grid interface is drawn based on the second view control, and the second grid interface is drawn based on the first view control.

7. The method according to claim 6, characterized in that, The second view control inherits from the recycle bin view control, and the first view control inherits from the Android view control. The size of both the first view control and the second view control is greater than or equal to the screen size. The recycle bin view control has a table view, and the table view supports responding to user touch operations. The Android view control does not have a table view.

8. The method according to claim 7, characterized in that, The first view control supports at least two levels, namely N columns and S columns. The first grid interface includes P rows of grids, and the second grid interface includes Q rows of grids, where P and Q are positive integers. After displaying the first grid interface, the method further includes: Based on the adaptation position values ​​of the images in the first grid interface, a second stitched image is obtained, wherein each row of the second stitched image includes 2S-1 images, and each column of the second stitched image includes T images. 2-(M) P / S) images, the adaptation position values ​​of adjacent grids in each row of the second stitched image differ by 1, and the adaptation position value of the image in the j-th row and i-th grid of the first stitched image differs by S from the adaptation position value of the image in the (j+1)-th row and i-th grid, where S is an integer greater than N and T is an integer greater than Q.

9. The method according to claim 8, characterized in that, After displaying the second grid interface corresponding to the first image, the method further includes: The system receives a third operation from the user on the electronic device, whereby the user's two fingers do not leave the screen between the second and third operations. In response to the third operation, a second image is cropped from the second stitched image, and a fourth grid interface corresponding to the second image is displayed, wherein each row of the second image includes S images, each column of the second image includes T images, and the fourth grid interface includes S columns and T rows of grids.

10. The method according to any one of claims 1 to 5, characterized in that, In the first grid interface, each grid displays one image; or, In the first grid interface, each grid displays a third image, which is a cropped image from a third stitched image. Each row of the third image contains M images, each column contains P images, and each row of the third stitched image contains 2M-1 images. The adaptation position values ​​of adjacent grids in each row of the third stitched image differ by 1, and the adaptation position value of the image in the j-th row and i-th grid of the third stitched image differs by M from the adaptation position value of the image in the (j+1)-th row and i-th grid.

11. The method according to any one of claims 1 to 5, characterized in that, After displaying the second grid interface corresponding to the first image, the method further includes: Based on the adaptation position values ​​of the images in the second grid interface, a fourth stitched image is obtained. Each row of the fourth stitched image includes 2K-1 images. The adaptation position values ​​of adjacent grid images in each row of the fourth stitched image differ by 1. The adaptation position value of the image in the j-th row and i-th grid of the fourth stitched image differs from the adaptation position value of the image in the (j+1)-th row and i-th grid by K, where K is not equal to N.

12. An electronic device, characterized in that, The electronic device includes: one or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 11.

13. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 11.

Images