Grid interface-based decoding method, device, chip system and medium

Abstract

This application provides a decoding method, device, chip system, and medium based on a grid-based interface, relating to the field of terminal technology. In this application, when a user performs a zoom or swipe operation on the screen, a new cache range is first estimated based on the current display area of ​​the View control; then, the new cache range is compared with the current cache range in the cache pool to determine the newly added image; the decoded newly added image is stored in the cache pool. Based on the updated data in the cache pool, the next frame of the grid interface is drawn in the View control. During the drawing cycle of each grid interface frame, the images that may be used in the next frame are estimated in advance, requiring only the newly added images to be decoded and stored in the cache pool, thus reducing the number of images to be decoded. When drawing the next frame of the grid interface, the decoded data is directly retrieved from the cache pool, reducing the probability that some grid images cannot be successfully decoded due to the image decoding speed not keeping up with the user's operation speed.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a decoding method, device, chip system and storage medium based on a grid interface. Background Technology

[0002] With the rapid development of terminal technology, mobile phone storage space is becoming increasingly larger. Mobile phone gallery apps can store a large number of images. Gallery apps can set up multiple albums based on image type, such as camera album, all photos album, all videos album, screenshot album, and personal favorites album, etc.

[0003] Currently, mobile phones support displaying a fixed number of columns of images in a grid format within each album's interface. When the user swipes or zooms on the screen, the phone needs to decode the images in each grid and redraw them in each grid to update the grid interface.

[0004] However, when users operate quickly, the phone's image decoding speed may not keep up, causing some grid images to fail to be decoded successfully, resulting in some grids on the screen having no image. This causes the user to see a blank grid interface, affecting the user experience. Summary of the Invention

[0005] This application provides a decoding method, device, chip system, and medium based on a grid interface, which can solve the problem of blank grid interfaces caused by fast user operation speed.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions: Firstly, embodiments of this application provide a method for displaying a grid interface. This method may include: The system receives a first user action; in response to the first action, it displays a first grid interface and obtains a first batch of decoded images based on the adaptation position values ​​of the images in the first grid interface. The first grid interface may include a grid with a first number of columns. The system also receives a second user action; in response to the second action, it obtains decoded images from the first batch of decoded images, representing the transition from the first adaptation position value to the second adaptation position value. Based on these decoded images, it draws and displays a second grid interface, which may include a grid with a second number of columns.

[0007] In the above solution, when a user touches the screen, the images that might be used in the next grid layout can be predicted in advance based on the adaptation position values ​​of the images in the current grid interface. This allows the decoded images to be directly retrieved from the batch-decoded images when rendering the next grid layout, reducing the possibility that some grid images might fail to be decoded due to the image decoding speed not keeping up with the user's operation speed. From the user's perspective, the updated grid interface displays images in each grid, thus improving the user experience.

[0008] In one possible implementation, the first and second grid interfaces are drawn based on a first view control, the size of which is larger than the screen size. The first view control supports scrolling and zooming. The first view control inherits from Android's view controls or from the recycle bin view control. Android's view controls do not have a table view, while the recycle bin view control does, and the table view supports responding to user touch operations.

[0009] In one possible implementation, the method can be applied to a sliding operation scenario, where the first and second operations are sliding operations, and the number of columns in the first column is the same as the number of columns in the second column. The method can also be applied to a scaling operation scenario, where the first and second operations are scaling operations, and the number of columns in the first column is different from the number of columns in the second column.

[0010] In one possible implementation, if the first operation and the second operation are sliding operations, obtaining the first batch of decoded images based on the adaptation position values ​​of the images in the first grid interface may include: calculating the adaptation position value of the grid located in the center of the first grid interface based on the adaptation position values ​​of the first grid and the last grid in the first grid interface; determining a first cache range based on the adaptation position value of the grid located in the center of the first grid interface and a first value, wherein the first value is the total number of grids that can be included on the screen when displaying a grid interface with a first column number; and obtaining the first batch of decoded images based on the first cache range.

[0011] For example, if the AP in the first cell of the display area is represented as windowStartAP, and the AP in the last cell of the display area is represented as windowEndAP, then the center AP can be represented as: centerAP=(windowStartAP+windowEndAP) / 2.

[0012] After obtaining the total number of grids in the screen display area at the current gear level, the center AP can be subtracted from the total number of grids to obtain the AP of the first image in the new cache range; and / or, the center AP can be added to the total number of grids to obtain the AP of the last image in the new cache range.

[0013] In the above scheme, during a swipe operation scenario, when the electronic device receives a swipe from the user, it triggers the rendering of the grid interface. Since the image that may be used for the next frame of the grid interface is obtained in advance, the image used to render the current frame can be directly obtained from the decoded image when rendering the next frame of the grid interface.

[0014] In one possible implementation, if the first operation and the second operation are sliding operations, before obtaining the decoded image from the first batch of decoded images from the first adaptation position value to the second adaptation position value, the method may further include: determining the number of rows moved by the first view control based on the sliding distance of the first operation; determining the first adaptation position value based on the number of rows moved by the first view control, the number of the first column, and the adaptation position value of the first cell in the first grid interface; and determining the second adaptation position value based on the number of rows moved by the first view control, the number of the first column, and the adaptation position value of the last cell in the first grid interface.

[0015] For example, an electronic device can determine the adapted position values ​​of the first and last grid cells in the display area of ​​the screen based on the sliding distance dy of the first view control, according to the formula: AP + s * g. Here, s represents the number of rows the first view control moves, g represents the current position of the view control (i.e., the first column number), and AP represents the adapted position value of the grid cell in the first grid cell interface before receiving the second operation.

[0016] In the above scheme, during a swipe operation scenario, when the electronic device receives a swipe operation from the user, it triggers the rendering of the grid interface. Since the grid interface scrolls up and down, the number of columns in each grid interface is equal before and after the screen refresh. Therefore, the adaptation position value from the first grid to the last grid in the second grid interface can be calculated based on the number of rows moved by the first view control and the number of columns in the first grid interface.

[0017] In one possible implementation, if the first and second operations are scaling operations, obtaining the first batch of decoded images based on the adaptation position values ​​of the images in the first grid interface may include: determining the adaptation position value of the selected grid in the first grid interface based on the coordinates of the center of the two fingers, where the coordinates of the center of the two fingers are determined based on the coordinates when the user presses down with two fingers, and the user's two fingers have not left the screen from the time of the two-finger press operation until the first and second operations; determining a first cache range based on the adaptation position value of the selected grid in the first grid interface based on the coordinates of the center of the two fingers, and a second value, where the second value is the total number of grids that can be included on the screen when displaying a grid interface with a second column number; and obtaining the first batch of decoded images based on the first cache range.

[0018] In the above scheme, when the electronic device receives a user's zoom operation, it estimates the next zoom level and the corresponding cache range (i.e., the images that may be used when drawing the next zoom level grid) based on the currently drawn grid interface, and pre-decodes and stores them in the cache pool. Because the images that may be used in the next grid frame are pre-stored in the cache pool, the decoded data can be directly retrieved from the cache pool when drawing the next grid frame, thus reducing the probability that some grid images cannot be successfully decoded due to the image decoding speed not keeping up with the zoom operation speed.

[0019] In one possible implementation, if the first and second operations are scaling operations, before obtaining the decoded image from the first batch of decoded images to the second adaptation position value, the method may further include: selecting a grid in the first grid interface based on the coordinates of the center of the two fingers; determining the drawing area of ​​the second grid interface in the first view control, wherein the coordinates of the center of the two fingers are determined based on the coordinates when the user presses down with two fingers; determining the first adaptation position value and the second adaptation position value based on the layout position value and adaptation position value of the grid selected in the first grid interface based on the coordinates of the center of the two fingers; wherein the first adaptation position value is the adaptation position value of the first grid in the drawing area, and the second adaptation position value is the adaptation position value of the last grid in the drawing area.

[0020] In the above scheme, when the electronic device receives the user's zoom operation, if the gear switching condition is met, it can determine the row and column of the area to be drawn in the first view control, as well as the AP value corresponding to each grid in the drawing area, based on the center of the two fingers and the zoom factor.

[0021] In one possible implementation, regardless of whether it's a zooming or sliding scene, obtaining the first batch of decoded images based on the first cache range can include: comparing the first cache range and the second cache range, where the second cache range is the range of decoded images stored in the cache pool before receiving the first operation, and the decoded images in the first and second cache ranges have the same size; if there are overlapping decoded images in the first and second cache ranges, clearing the decoded images that exceed the first cache range from the cache pool and storing the non-overlapping decoded images in the cache pool to obtain the first batch of decoded images; if there are no overlapping decoded images in the first and second cache ranges, clearing all decoded images in the second cache range from the cache pool and storing all decoded images in the first cache range in the cache pool to obtain the first batch of decoded images.

[0022] In the above scheme, the new cache range and the current cache range may or may not overlap. When the new cache range and the current cache range overlap, the image loader needs to clear images that are outside the new cache range and store them in the cache range that needs to be updated. When the new cache range and the current cache range do not overlap, the image loader needs to clear all images in the current cache range and store all images from the new cache range in the cache pool.

[0023] In one possible implementation, regardless of whether it's a zooming or sliding scene, after receiving the second operation, the method may further include: receiving the user's hand-raising operation; responding to the hand-raising operation, determining whether the size of each image in the decoded image from the first adaptation position value to the second adaptation position value is smaller than a first size; if the size of the first image is smaller than the first size, initiating a decoding task for the first image to obtain a second image of the first size; replacing the first image with the second image to draw a third grid interface; and displaying the third grid interface. Here, the content of the first and second images is the same, and the first size corresponds to the number of the second column.

[0024] In the above scheme, after a rapid swipe or zoom operation, the electronic device usually remains on the last grid screen for a relatively long time. In this case, if the grid screen uses a low-sized bitmap, it may be noticeable to the user. By redrawing the last grid screen, the bitmap size of each cell in the last grid screen can be made larger, thus making the grid screen appear clearer to the user after scrolling or zooming.

[0025] In one possible implementation, the size of the first batch of decoded images is a first size, which corresponds to the number of columns in the second grid. Accordingly, obtaining decoded images from the first adaptation position value to the second adaptation position value from the first batch of decoded images, and drawing the second grid interface based on these decoded images, can include: for each adaptation position value from the first to the second adaptation position value, performing the following process: determining whether the first batch of decoded images can include a decoded image corresponding to an adaptation position value; if the first batch of decoded images can include a decoded image corresponding to an adaptation position value, then drawing the second grid interface based on the decoded image corresponding to that adaptation position value in the first batch of decoded images; if the first batch of decoded images cannot include a decoded image corresponding to an adaptation position value, then drawing the second grid interface based on the decoded image corresponding to that adaptation position value in the second batch of decoded images, where the size of the second batch of decoded images is a second size, which is smaller than the first size.

[0026] In the above solution, prioritizing the retrieval of bitmaps from a cache pool that meets the required size ensures image clarity. When a bitmap cannot be retrieved from a cache pool that meets the required size, retrieving a lower-sized bitmap avoids blank areas in the grid interface. In scenarios involving rapid scrolling or zooming, due to the high refresh rate of the grid interface, the use of lower-sized bitmaps in some grid cells is generally imperceptible to the user.

[0027] In one possible implementation, before receiving the user's first operation, the method may further include: receiving the user's third operation on the album card; and in response to the third operation, creating a first suspended task, N second suspended tasks, and N cache pools. The first suspended task is used to handle the decoding task of a single image, each of the N second suspended tasks is used to handle the decoding task of multiple images of one size, and each of the N cache pools is used to store decoded images of one size.

[0028] In the above scheme, when a user opens a photo album in the gallery app, the gallery app executes the initialization process corresponding to that album, creating a suspended thread for batch decoding, a suspended thread for single decoding, and cache pools corresponding to images of different sizes. This allows the gallery app to directly call these threads to perform the corresponding decoding tasks when the user performs swiping or zooming operations on the screen.

[0029] Secondly, this application provides an apparatus comprising units for performing the method described in the first aspect above. This apparatus corresponds to performing the method described in the first aspect above; for details regarding the units within the apparatus, please refer to the description in the first aspect above, which will not be repeated here for brevity. The method described in the first aspect above can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the aforementioned functions. For example, a processing module or unit, a display module or unit, etc.

[0030] Thirdly, this application provides an electronic device comprising: one or more processors, and a memory. The memory is coupled to the one or more processors and is used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the electronic device to perform the methods provided by the first aspect and any possible implementation thereof.

[0031] Fourthly, this application provides a computer-readable storage medium. The computer-readable storage medium includes computer instructions. When executed on an electronic device, the computer instructions cause the electronic device to perform the method provided by the first aspect and any possible implementation thereof.

[0032] Fifthly, this application provides a computer program product. When the computer program product is run on a computer, it causes the computer to perform the method provided by the first aspect and any possible implementation thereof.

[0033] Sixthly, this application provides a chip system applied to an electronic device, the chip system including one or more processors, the one or more processors being configured to invoke computer instructions to cause the electronic device to perform the methods provided in the first aspect and any possible implementation thereof.

[0034] It is understood that the beneficial effects achieved by the apparatus of the second aspect, the electronic device of the third aspect, the computer-readable storage medium of the fourth aspect, the computer program product of the fifth aspect, and the chip system of the sixth aspect provided above can be referred to as the beneficial effects of the first aspect and any possible implementation thereof, which will not be repeated here. Attached Figure Description

[0035] Figure 1 One of the schematic diagrams illustrating a scenario where a mobile phone displays images in a zoomed-out format on a photo album page, as provided in this application; Figure 2 The second illustration of a scenario where a mobile phone displays images in a zoomed-out format on a photo album page, as provided in this application; Figure 3 One of the schematic diagrams illustrating a scenario where a mobile phone displays images scrolling through a photo album page, as provided in this application; Figure 4 The second illustration of a scenario where images are scrolling through a photo album page on a mobile phone, as provided in this application; Figure 5 This is a schematic diagram showing that some grids in the grid interface provided in this application do not have images. Figure 6 A schematic diagram of a low-density view provided in this application; Figure 7 A schematic diagram illustrating a cache range update strategy provided in this application; Figure 8 A schematic diagram illustrating another cache range update strategy provided in this application; Figure 9 A schematic diagram of a cache pool provided in this application; Figure 10 A schematic diagram illustrating that multiple photo albums provided in this application share a single cache pool; Figure 11 A software architecture diagram of an electronic device provided in this application; Figure 12 A schematic diagram illustrating the image decoding scheme involved in the grid interface provided in this application; Figure 13A schematic diagram illustrating the relationship between batch decoding data and the grid drawing interface in each drawing cycle, as provided in this application; Figure 14 A schematic diagram illustrating an initialization process corresponding to a photo album, provided for the purposes of this application; Figure 15 A schematic diagram illustrating the overall process of drawing a grid interface in a sliding operation scenario provided in this application; Figure 16 This is a schematic diagram illustrating the method flow for updating the cache pool in a sliding operation scenario provided in this application. Figure 17 A schematic diagram illustrating the updating of the cache in a downward scrolling scenario provided in this application; Figure 18 A schematic diagram illustrating the updating of the cache in an upward swiping scenario provided in this application; Figure 19 A schematic diagram illustrating the overall process of drawing a grid interface in a zoom operation scenario provided in this application; Figure 20 A schematic diagram of each grid AP in the display area before and after the scaling operation provided in this application; Figure 21 This is a schematic diagram illustrating the method flow for updating the cache pool in a scaling operation scenario provided in this application; Figure 22 A schematic diagram illustrating the priority of obtaining bitmaps from a cache pool that meets the usage size, as provided in this application; Figure 23 A schematic diagram illustrating the decoding method for a single image in the last frame of the grid interface provided in this application; Figure 24 A schematic diagram of the redrawn last stitch grid interface provided for this application; Figure 25 This is a schematic diagram of the hardware structure of an electronic device provided in this application. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application.

[0037] In this embodiment of the application, the gallery application supports multiple image display methods. For example, such as... Figures 1 to 4 As shown, the gallery app supports categorizing and displaying images in album format on the album page. The album page can include various types of albums such as camera, video, photo, cloud storage, selfie, live photo, long exposure, slow motion, screenshot, and GIF.

[0038] For photo album pages, electronic devices have multiple preset levels. A "level" can be understood as the number of image columns displayed on the screen. For example, electronic devices may have preset levels of 3, 5, 13, and 31 for photo album pages. Level 3 is the minimum level, and level 31 is the maximum. At level 3, 3 columns of images are displayed on the screen; at level 5, 5 columns; at level 13, 13 columns; and at level 31, 31 columns.

[0039] Users can use a two-finger zoom gesture to change the number of image columns displayed on the screen for a photo album page. As an example, by decreasing the distance between two fingers, the user can trigger the device to increase the number of image columns displayed on the album page. Conversely, by increasing the distance between two fingers, the user can trigger the device to decrease the number of image columns displayed on the album page. For instance, if the user sets the album page to level 3 using a zoom gesture, the device will maintain a display of 3 columns of images until the next two-finger gesture is received.

[0040] The following example uses a mobile phone as an electronic device, combined with... Figures 1 to 4 This section provides examples illustrating the use cases of image library applications.

[0041] In some embodiments, the phone supports zooming in and out of images on the photo album page in response to two-finger gestures.

[0042] For example, Figure 1 and Figure 2 This illustration shows a scenario where a mobile phone zooms in to display images on a photo album page.

[0043] Figure 1 and Figure 2 The example provided uses 3, 5, 13, and 31 preset levels for the photo album page, which is not intended to limit this application. In actual implementation, the phone can arbitrarily set the number of levels for the photo album page. For example, based on the phone screen size, 1, 3, 5, 13, 21, and 31 levels can be preset for the photo album page, with level 1 being the minimum and level 31 being the maximum. Users can switch between these six levels by using a two-finger zoom gesture.

[0044] like Figure 1 As shown in (a), the phone displays icons for applications such as the Gallery app on the home screen. When a user wants to view images, they can tap the Gallery icon 01. In response to the user's tap on the Gallery icon 01, the phone displays... Figure 1The card page is shown in (b). The card page comprises several areas: a status bar displaying battery and signal strength indicators, an action bar providing photos and a search box, and a foot bar showcasing photos, albums, memories, and creations. When an album in the foot bar is selected, the central area of ​​the card page displays album cards for the camera, all photos, and videos. These album cards, also known as photo album cards, each serve as an entry point to an album.

[0045] Users can select an album card from these album cards. For example, ... Figure 1 As shown in (b), the user can tap the "Camera" card 02. In response to the user's tap on the "Camera" card 02, the phone displays the following... Figure 1 The album page shown in (c) is an example. The album page comprises multiple square grids. Taking an album page with 3 grids as an example, the side length h1 of each grid is equal to the screen width divided by 3. Each grid displays a single image, which can be a thumbnail of a photograph or a video clip. Additionally, when the album page has 3 grids, a time capsule 03, displayed in days with preset transparency, is shown in the upper left corner of the album page; at this point, the album page is called a day view. As an example, the time capsule provided in this embodiment is a control or prompt that floats above the grid interface, indicating the shooting date of the images in the grid interface. For example, time capsule 03 indicates the specific shooting date "2024.07.10," which could be the shooting date of all photos in the current album page. When a user wants to quickly browse more photos, the user can reduce the distance between two fingers to trigger the phone to increase the number of image columns displayed on the screen.

[0046] like Figure 1 As shown in (d), the user can press two fingers, at which point the distance between the two fingers is d1. Figure 1 As shown in (e), without lifting two fingers from the screen, the user can decrease the distance between their fingers. When the distance decreases from d1 to d2, the album page switches from a 3-column grid to a 5-column grid, with each grid having a side length of h2, which is equal to the screen width divided by 5. When the album page includes 5 columns, a time capsule 04, displayed in days, is shown in the upper left corner with preset transparency. For example, the specific shooting date indicated by time capsule 04 is still "2024.07.10", which could be the shooting date of all photos in the current album page. At this point, the album page is still called the day view.

[0047] like Figure 1As shown in (f), without lifting two fingers from the screen, the user can continue to decrease the distance between their fingers. When the distance decreases from d2 to d3, the album page switches from a 5-column grid to a 13-column grid, with each grid having a side length of h3, which is equal to the screen width divided by 13. When the album page includes 13 columns, two time capsules, 05 and 06, are displayed on the left side of the album page with preset transparency, each representing a month. The photos in the grid between time capsules 05 and 06 are dated "July 2024," while the photos in the rows containing time capsule 06 and below are dated "August 2024." At this point, the album page is called the Month View.

[0048] like Figure 2 As shown in (a), without lifting two fingers from the screen, the user can continue to decrease the distance between their fingers. When the distance decreases from d3 to d4, the album page changes from a 13-column grid to a 31-column grid, with each grid having a side length of h4, which is equal to the screen width divided by 31. When the album page includes 31 columns, a time capsule 07 in years is displayed in the upper left corner of the album page with preset transparency. For example, time capsule 07 indicates a specific shooting date of "2024," which could be the shooting date of all photos in the current album page. At this point, the album page is called the year view.

[0049] As an example, after the album page is switched to a 31-column grid, such as Figure 2 As shown in (b), if the user raises their hand, the album page continues to display 31 columns of images.

[0050] As another example, after switching the album page to a 31-column grid, such as Figure 2 As shown in (c), if the user keeps the distance between their two fingers constant at d4, the album page will continue to display 31 columns of images.

[0051] As another example, after switching the album page to a 31-column grid, such as Figure 2As shown in (d), if the user increases the spacing between two fingers, for example, from d4 to d5, the album page switches from a 31-column grid to a 13-column grid. When the album page includes 13 columns, two time capsules, 08 and 09, are displayed on the left side of the album page with preset transparency, each representing a month. The photos in the cell between Time Capsule 08 and Time Capsule 09 were taken on "July 2024," while the photos in the cells of Time Capsule 09 and the cells below it were taken on "August 2024." It can be understood that the user can continue to increase the spacing between two fingers to trigger the phone to increase the number of columns in the grid. For example, if the user continues to increase the spacing between two fingers, the album page switches from a 13-column grid to a 5-column grid; subsequently, if the user continues to increase the spacing between two fingers, the album page switches from a 5-column grid to a 3-column grid.

[0052] It should be noted that this application does not limit the number of image columns displayed when a mobile phone enters a photo album page. For example, the number of image columns displayed each time a mobile phone enters a photo album page can be as follows: Figure 1 The three columns shown in (c) can also be as follows: Figure 1 The five columns shown in (e) are an example. For instance, the number of image columns when the phone enters the album page this time is the same as the number of image columns when it exits the album page last time.

[0053] In other embodiments, the phone also supports scrolling images in response to a single finger gesture on the photo album page.

[0054] For example, Figure 3 and Figure 4 This illustration shows a scenario where a mobile phone scrolls through images on a photo album page.

[0055] Users can swipe down to trigger the phone to display more images taken earlier. For example... Figure 3 As shown in (a), the album page includes two time capsules, 10 and 11, each representing a month. Time capsule 10 indicates a shooting date of "July 2024," and time capsule 11 indicates a shooting date of "August 2024." Users can press and hold image 455 with a single finger and then swipe down. Figure 3 As shown in (b), if a single finger slides down a distance d6, in response to the user's swipe, the album page slides down a distance d6. If all photos on the current page were taken in July 2024, then the specific shooting date indicated by Time Capsule 12 is "July 2024". Figure 3As shown in (c), the user can continue to press and hold image 455 and slide it down a distance d7. In response to the user's sliding action, the album page slides down a distance d7. If all photos on the current page were taken in June 2024, then the specific shooting date indicated by Time Capsule 13 is "2024.06". Typically, the sliding distance of the downward follow-up sliding action is equal to the distance the album page scrolls down. The longer the downward follow-up sliding action, the more content is displayed when the album page scrolls down.

[0056] Users can swipe up to trigger the phone to display more images taken later. For example... Figure 4 As shown in (a), the specific shooting date indicated by Time Capsule 14 on the album page is "2024.02". Users can press and hold image 350 with a single finger and then slide the finger upwards. (As shown in the image) Figure 4 As shown in (b), if a single finger slides upwards a distance d8, in response to the user's sliding action, the album page slides upwards a distance d8. If all photos on the current page were taken in March 2024, then the specific shooting date indicated by Time Capsule 15 is "March 2024". Figure 4 As shown in (c), the user can continue to press and hold image 350 and slide it upwards a distance d9. In response to the user's swipe, the album page slides upwards a distance d9. At this moment, the current page includes two time capsules, 16 and 17, each representing a month. The photos in the grid between time capsules 16 and 17 were taken on "March 2024," while the photos in the grids of time capsule 17 (both in the row and below) were taken on "April 2024." Typically, the upward swipe distance is equal to the distance the album page scrolls upwards. The longer the upward swipe distance, the more content is displayed when the album page scrolls upwards.

[0057] In the above embodiments, the album page includes multiple grids, each grid displaying a texture image. It should be noted that, due to interface size limitations, the embodiments in this application only show texture images in a grid at level 3. It is understood that as the level increases, each grid should include a smaller texture image.

[0058] As described in the above embodiments, the mobile phone supports displaying a grid interface in response to two-finger zoom or single-finger swipe operations. When the user performs a two-finger zoom or single-finger swipe operation on the screen, the mobile phone needs to decode the image of each grid and redraw the image in each grid to update the grid interface.

[0059] However, when users perform two-finger zoom or single-finger swipe operations quickly, the phone's image decoding speed may not keep up with the user's operation speed due to processor performance limitations. This can cause some grid images to fail to be decoded, resulting in some grids of the grid interface having no image. For example, ... Figure 5 As shown in (a), when the phone displays a 3-level grid interface, the user quickly swipes one finger upwards on the screen; in response to the swipe operation, as shown in (a), the screen displays a grid interface. Figure 5 As shown in (b), the phone updates the grid interface of level 3, but some of the images in the grid interface are not loaded. For example, as... Figure 5 As shown in (c), when the phone displays a 3-level grid interface, the user quickly reduces the distance between two fingers on the screen; in response to the zoom operation, as... Figure 5 As shown in (d), the phone switches from a 3-level grid interface to a 5-level grid interface, but some images in the 5-level grid interface fail to load. From the user's perspective, the updated grid interface appears blank, thus affecting the user's operating experience.

[0060] In view of the above problems, this application provides a scheme for decoding images involved in a grid interface based on a View control. The View control can be considered a container for displaying images. The electronic device can create a cache pool for storing images. When the user zooms or swipes the screen, the electronic device can first estimate a new cache range based on the current display area of ​​the View control; then compare the new cache range with the current cache range in the cache pool to determine the newly added image, and decode the newly added image; after decoding, the decoded data of the newly added image is stored in the cache pool. In this way, the electronic device can draw the next frame of the grid interface in the View control based on the updated data in the cache pool. In the above process, during the drawing cycle of each grid interface frame, the images that may be used in the next frame of the grid interface are estimated in advance, and only the newly added images need to be decoded and stored in the cache pool, thus reducing the number of images to be decoded. When drawing the next frame of the grid interface, the electronic device can directly obtain the decoded data (also called decoded images) from the cache pool, reducing the probability that some grid images cannot be successfully decoded due to the image decoding speed not keeping up with the user's operation speed. From the user's perspective, the updated grid interface displays images in each grid, thereby improving the user's operating experience.

[0061] It should be noted that the above solutions can be based on Android electronic devices, or on electronic devices based on other operating systems. These electronic devices are also referred to as terminals or user equipment (UE). For example, electronic devices can be personal computers (PCs), mobile phones, smart screens, smart TVs, tablets, wearable devices, computers with wireless transceiver capabilities, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, etc., or they can be other devices or apparatuses with image library functionality.

[0062] The View control will be introduced below.

[0063] This application provides two types of View controls: low-density View and high-density View. The low-density View inherits from the RecyclerView control, while the high-density View inherits from an Android-based View control. RecyclerView and Android-based View controls are two different types of View controls. RecyclerView is a special type of View control, composed of multiple ItemViews in a grid format. Each ItemView displays an image view, and each ItemView supports user touch operations on individual ItemViews. Android-based View controls are ordinary View controls, which can be considered a blank canvas and do not have the concept of ItemViews, but offer advantages in performance and memory efficiency. The individual grids in an Android View control are not containers and do not support user touch operations on individual ItemViews. In this application embodiment, the initialized sizes of the low-density View and high-density View can be equal or unequal. Both the low-density View and high-density View have extended functionality, supporting scrolling and zooming. For example, when a user swipes their finger up and down on the screen, the low-density view and the high-density view will scroll up and down accordingly, and the grid interface will also scroll up and down accordingly; or, when a user zooms in or out on the grid interface, the low-density view and the high-density view will zoom in or out accordingly, and the grid interface will also zoom in or out accordingly.

[0064] For scenarios with a small number of images to display, such as a grid interface with 3 or 5 columns of images, electronic devices can draw relevant feature icons (also known as controls) on the thumbnails corresponding to the grid based on the business status of the photos and videos. Figure 1 The cloud control, favorites control, or duration control shown in (c) are examples of such controls. In some scenarios, electronic devices need to support responding to user clicks on a control to achieve the corresponding function. However, Android's View control does not support touch operations on controls and therefore cannot achieve the corresponding function. For this reason, this application provides a grid interface display solution based on a custom RecyclerView control (i.e., a low-density View).

[0065] For scenarios involving a large number of images to be displayed, such as a grid interface with 13 or 31 columns of images, electronic devices do not need to draw controls on the thumbnails. In this case, the electronic device does not need to support responding to user clicks on any control. Android's View controls can fully meet the display function of a grid interface, and the implementation is simpler. For this reason, this application also provides a grid interface display solution based on Android's View controls (i.e., high-density Views).

[0066] During the display of the grid interface, the electronic device can respond to the user's zooming operation, switching between a low-density view and a high-density view to change the number of grid columns in the current grid interface. Both low-density and high-density views involve two indices: one is the adapter position (AP), and the other is the layout position (LP). AP is an index specifically set from the adapter's perspective for the content displayed on the screen, representing the number of the image displayed in the grid. LP is an index established from the layout manager's perspective, representing the position of the grid within the view control. In the native logic of the Android system, LP cannot be customized, but in this embodiment, the electronic device can change the specific value of LP based on zoom level, two-finger center gestures, scrolling operations, etc. In the low-density view, LP is consecutively numbered from the first row and first column of the entire low-density view to the last row and last column of the control's low-density view. In a high-density view, LP is numbered consecutively from the first row and first column of the drawing area of ​​the high-density view to the last row and last column of the drawing area of ​​the high-density view.

[0067] For example, Figure 6 This is a schematic diagram of a low-density view provided in this application.

[0068] When the album window needs to display 3 or 5 columns of images, the total number of images in the album window is relatively small, so a single low-density view can be used. If the scaling factor of the low-density view is set to 1.0 when the album window displays 3 columns, then when the album window displays 5 columns, the low-density view needs to be shrunk by 3 / 5 = 0.6 times. Based on this, the size of the low-density view is calculated as follows: When the album window needs to display 3 columns of images, divide the width of the album window (1300 pixels) by 3 columns to get the width / height of each grid cell as 434 pixels; divide the height of the album window (2875 pixels) by the height of each grid cell as 434 pixels to get the number of rows of images that the album window needs to display as 7 + 1 = 8 rows; multiply the width of each grid cell (434 pixels) by 5 columns to get the width of the low-density view as 2170 pixels; multiply the width of the low-density view (2170 pixels) by the height of the album window (2875 pixels), then divide by the width of the album window (1300 pixels), then divide by the height of each grid cell as 434 pixels to get the number of rows of the low-density view as 12 + 1 = 13 rows. Thus, the low-density view consists of 5 columns * 13 rows of grid cells, with each grid cell having a width of 434 * a height of 434 pixels. Figure 6As shown, when the scaling factor of the low-density view is set to 1.0, the drawing area (3 columns * 8 rows of grid) of the low-density view is sent to the display screen at a scaling factor of 1.0. In the drawing area of ​​the low-density view, each grid is used to display a texture image, and each texture image corresponds to an AP value. The adaptation position values ​​corresponding to each grid in the drawing area increase sequentially from left to right and from top to bottom. For example, in the drawing area, the AP of the first row and first column is i, the AP of the first row and second column is i+1, the AP of the first row and third column is i+2, the AP of the second row and first column is i+3, and so on. It should be noted that, Figure 6 This explanation uses the AP encoding method for low-density views as an example. The AP encoding method for high-density views is similar and will not be repeated here. Figure 6 In the image, the actual number of rows displayed is one more than the number of rows allowed on the screen. This is to allow the view to scroll when the user swipes their finger up and down on the screen without leaving any blank space.

[0069] The following is a brief explanation of the basic principles of image decoding and caching.

[0070] In this embodiment of the application, when a user performs a zoom or swipe operation on the screen, the electronic device can estimate a new cache range; then compare the new cache range with the current cache range in the cache pool to determine the overlapping data in the two cache ranges, thereby determining the cache range that needs to be updated; then decode the cache range that needs to be updated; after decoding is completed, store the decoded data (also known as the decoded image) in the cache pool.

[0071] For example, Figure 7 A schematic diagram of a cache range update strategy is shown. For example... Figure 7 As shown, when a user performs a two-finger zoom or a single-finger swipe on the screen, the AP of the first image in the cache pool is represented as currentStartAP=20, and the AP of the last image in the cache pool is represented as currentEndAP=62, meaning the current cache range is [20, 62]. The electronic device can estimate a new cache range based on the two-finger zoom or single-finger swipe operation, such as [52, 94]. Since there is an overlap between the current cache range and the new cache range in the cache range [52, 62], the electronic device does not need to update all images in the cache pool; it only needs to clear images that exceed the new cache range and store them in the cache range that needs to be updated. For example, the AP of the first image in the updated cache range is represented as updateStartAP=63, and the AP of the last image in the updated cache range is represented as updateEndAP=94, meaning the updated cache range is [63, 94].

[0072] For example, Figure 8 A diagram illustrating another cache range update strategy is shown. For example... Figure 8 As shown, when a user performs a two-finger zoom or a single-finger swipe on the screen, the AP of the first image in the cache pool is represented as currentStartAP=20, and the AP of the last image in the cache pool is represented as currentEndAP=62, meaning the current cache range is [20, 62]. The electronic device can estimate a new cache range based on the two-finger zoom or single-finger swipe operation, such as [80, 122]. Since there is no overlap between the current cache range and the new cache range, the electronic device needs to clear all images in the current cache range and store images from the new cache range in the cache pool. For example, the AP of the first image in the updated cache range is represented as updateStartAP=80, and the AP of the last image in the updated cache range is represented as updateEndAP=122, meaning the updated cache range is [80, 122].

[0073] Generally, larger image sizes result in higher image clarity, but also consume more memory. As described in the above embodiments, the electronic device pre-sets multiple tiers for the photo album page. With each tier, the size of a single grid in the interface decreases, and the number of grids in a single grid interface increases. If all grids across all tiers use large-sized images, it will severely strain memory and may even cause the electronic device to malfunction. Research has found that larger grid sizes indicate higher user requirements for image clarity, while smaller grid sizes indicate lower user requirements for image clarity. Based on user requirements for image clarity, this application proposes creating multiple cache pools (also called caches), each cache pool storing images of a specific size corresponding to a tier. When the cache range for a tier is updated, the electronic device can store the decoded image in the cache pool corresponding to that tier. Accordingly, when rendering a grid interface for a tier, the electronic device can read the image from the cache pool corresponding to that tier.

[0074] For example, Figure 9 A schematic diagram of a cache pool is shown. (For example...) Figure 9As shown, the electronic device pre-creates three buffer pools: a 512-pixel buffer pool, a 256-pixel buffer pool, and a 96-pixel buffer pool. The 512-pixel buffer pool stores images with a width of 512 pixels and a height of 512 pixels; the 256-pixel buffer pool stores images with a width of 256 pixels and a height of 256 pixels; and the 96-pixel buffer pool stores images with a width of 96 pixels and a height of 96 pixels. An example is shown with a display setting of 512 pixels (5x512 pixels), a display setting of 256 pixels (256 pixels), and a display setting of 96 pixels (96 pixels). When the cache range of level 3 is updated, the electronic device calls the decoding service through a 512-level decoding pipeline, storing the newly loaded decoding data corresponding to level 3 into a 512-level cache pool. When the cache range of level 5 is updated, the electronic device calls the decoding service through a 256-level decoding pipeline, storing the newly loaded decoding data corresponding to level 5 into a 256-level cache pool. When the cache range of level 13 is updated, the electronic device calls the decoding service through a 96-level decoding pipeline, storing the newly loaded decoding data corresponding to level 13 into a 96-level cache pool. It can be understood that by setting separate cache pools for different levels, the memory consumption of the electronic device can be reduced while still meeting the user's needs for image clarity in different scenarios.

[0075] In some embodiments, due to the limited capacity of the decoding pipeline (such as the Kotlin pipeline), the decoding pipeline can discard some tasks when there is a backlog. For example, each decoding pipeline can support a maximum of two tasks. Figure 9 As shown, when the 512 decoding pipeline receives tasks a1, a2, and a3 in sequence, it can retain only the most recently received tasks a2 and a3, while discarding the earlier received task a1. The handling of tasks by the 256 and 96 decoding pipelines is similar and will not be elaborated upon here.

[0076] In some scenarios, users may open multiple albums in a gallery application. Typically, at any given time, a user will only zoom, swipe, or perform other operations on one album at a time. Therefore, to reduce the load on the cache pool, this application also proposes a scheme where multiple albums share a single cache pool. It should be noted that this "single cache pool" includes: one 512-pixel cache pool (referred to as the first cache pool), one 256-pixel cache pool (referred to as the second cache pool), and one 96-pixel cache pool (referred to as the third cache pool).

[0077] For example, Figure 10 This diagram illustrates a scenario where multiple albums share a single cache pool.

[0078] like Figure 10As shown, albums 1 through N share a 512-byte cache pool. When the cache range of a certain album (level 3) is updated, the decoding service can be called through the 512-byte decoding pipeline to store the newly loaded decoded data into the 512-byte cache pool. Albums 1 through N share a 256-byte cache pool. When the cache range of a certain album (level 5) is updated, the decoding service can be called through the 256-byte decoding pipeline to store the newly loaded decoded data into the 256-byte cache pool. Albums 1 through N share a 96-byte cache pool. When the cache range of a certain album (level 13) is updated, the decoding service can be called through the 96-byte decoding pipeline to store the newly loaded decoded data into the 96-byte cache pool. Here, N is an integer greater than or equal to 2.

[0079] In this embodiment, the software system of the electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, or microservice architecture. Taking the layered architecture Android system as an example, the software system architecture of the electronic device is illustrated.

[0080] For example, Figure 11 A software architecture diagram of an electronic device is shown.

[0081] like Figure 11 As shown, electronic devices can adopt a layered architecture, dividing the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the software layers of the software architecture are divided from top to bottom as follows: application (APP) layer, application framework (FWK) layer, system library, and kernel layer. This software architecture runs on top of the hardware layer, which may include graphics processing units (GPUs), displays, and sensors, etc.

[0082] The application layer may include a series of application packages, such as a camera application, a gallery application, a data loader, a bitmap loader, view controls (such as a custom RecyclerView), a custom DraweeView, and a hierarchy pool. The camera application can use the camera to capture photos and videos and store them in the gallery application. The gallery application can manage captured photos and videos, as well as screen captures and shared images. It should be noted that the data loader, bitmap loader, custom DraweeView, hierarchy pool, and view controls can be functional modules within the gallery application or independent of it. The specific functions of the data loader, bitmap loader, custom DraweeView, and hierarchy pool are described in the following embodiments and will not be repeated here.

[0083] When these application packages are run, they can access the various service modules provided by the application framework layer through the application programming interface (API) and execute corresponding intelligent business operations.

[0084] The application framework layer provides APIs and a programming framework for applications. It includes predefined functions. For example, the application framework layer may include a window manager service (WMS), an activity manager service (AMS), SurfaceFlinger, and a file manager. Specifically, WMS is used to pass data collected by the touch driver to the currently bound window control, such as a custom RecyclerView.

[0085] The system library can include multiple functional modules, such as the surface manager, media libraries, a 2D graphics engine (e.g., SGL), and a 3D graphics processing library (e.g., OpenGL ES). Within the system library, the Android Runtime includes the core libraries and the virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part contains the functionalities that the Java language needs to call, and the other part contains the core Android libraries. The application layer and application framework layer run in the virtual machine.

[0086] The kernel layer is the layer between hardware and software, and belongs to the lower level of the Android system. The kernel layer can contain various driver interfaces, such as display drivers, audio drivers, touch drivers, and sensor drivers.

[0087] It should be noted that although the embodiments of this application are illustrated using the Android system as an example, the basic principles are equally applicable to electronic devices based on operating systems such as iOS or Windows.

[0088] It is understood that, in order to implement the grid-based decoding method in the embodiments of this application, the electronic device includes hardware and / or software modules that perform various functions. Based on the algorithm steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments.

[0089] The following is combined Figure 12 A brief explanation of the overall logic of the image decoding scheme involved in the grid interface.

[0090] The image loader includes a settings center (setCenter), a batch decoding module, a Fresco submission module, a single decoding module, a cache pool, and a retrieval module. For example, the retrieval module could use the `get()` interface. Fresco is an image library from Facebook used to display images in Android applications. The Fresco submission module is a custom interface used to call Fresco's open-source code to provide decoding services.

[0091] like Figure 12 As shown, the process from when a user's finger lands on the screen to when the grid interface is displayed can be divided into three stages: I. Batch Decoding Stage.

[0092] When a user's finger lands on the screen, the touch driver notifies the gallery app. The gallery app can then calculate the AP (Area of ​​Action) of the center grid based on the type of touch operation, referred to as the center AP. As an example, in a swipe scenario, the center grid is the grid located in the center of the current screen area. As another example, in a zoom scenario, the center grid is the grid selected by two fingers. The gallery app passes the center AP to the image loader via the data loader. The image loader then performs batch decoding of the images.

[0093] Specifically, the image loader's settings center calculates the new cache range based on the central AP and submits a task list to the batch decoding module. A suspended task in the batch decoding module switches from a suspended state to an active state, checks the task list, and compares the new cache range with the current cache range to see if any new images have been added. If new images have been added, the suspended task creates a Fresco task and submits it to the Fresco submission module. The Fresco submission module submits a decoding request for the new images to the Fresco open-source code. After the Fresco open-source code completes decoding, it returns the decoded data to the Fresco submission module. The Fresco submission module then submits the decoded data to the cache pool corresponding to the suspended task and clears cached data that exceeds the new cache range. For example, if suspendable task 1 is used to process images of size 512, then the Fresco submission module will submit the decoded 512-size image to the 512 cache pool; if suspendable task 2 is used to process images of size 256, then the Fresco submission module will submit the decoded 256-size image to the 256 cache pool; if suspendable task 3 is used to process images of size 96, then the Fresco submission module will submit the decoded 96-size image to the 96 cache pool.

[0094] II. Single decoding stage.

[0095] When a user's two-finger zoom or single-finger swipe is fast, the processor's performance limitations may prevent the electronic device from decoding images quickly enough, resulting in some grid images failing to be decoded. When the electronic device tries to retrieve images from the cache pool via the `get()` interface, it fails to do so. In this case, the `get()` interface submits a single decoding task for the unretrievable image to a single decoding module, marking it as a callback. The single decoding module's suspended task 4 switches from suspended to active, creates a Fresco task, and submits the task to the Fresco submission module. The Fresco submission module submits a decoding request for the unretrievable image to the Fresco open-source code. After decoding, the Fresco open-source code returns the decoded data of the single image to the Fresco submission module. Since this image's decoded data has been marked as a callback, the Fresco submission module can return the decoded data to the `get()` interface via an asynchronous sub-thread callback function.

[0096] III. Drawing stage.

[0097] The drawing phase involves three functional modules: View controls (such as a custom RecyclerView), the granting object view (DraweeView), and the hierarchical pool. In one possible implementation, the custom DraweeView inherits from Fresco's SimpleDraweeview, converting each control into a Drawable object, constructing a list of Drawables, and then placing this list into Drawee's overlay. Leveraging Drawee's existing drawing process, the controls are drawn as a layer. In another possible implementation, the custom DraweeView does not use Fresco's mechanism. Instead, it directly inherits from Android's View control as the ItemView of the custom RecyclerView, creates a custom ItemHierarchy class, constructs a list of Drawables in the custom hierarchical pool, and adds each control as a Drawable to the list. During drawing, each Drawable control is drawn individually.

[0098] During the drawing phase, the View control (such as a custom RecyclerView) submits a drawing request to the DraweeView. The DraweeView retrieves the image to be drawn from the Hierachy pool. The Hierachy pool calls the get() interface to read data from the cache pool.

[0099] After the `get()` interface retrieves the batch decoded data from the cache pool, it returns it to the Hierarchy pool. The View control then draws the grid interface based on the batch decoded data. It should be noted that the View control will draw the grid interface based on the batch decoded data regardless of whether the image has been decoded, which may result in some grid cells appearing blank.

[0100] In scenarios where the `get()` interface cannot retrieve certain images from the cache pool, after the Fresco submission module returns the decoded data to the `get()` interface via an asynchronous sub-thread callback function, the `get()` interface returns a single decoded data entry to the Hierarchy pool. The View control then redraws a grid interface based on this single decoded data entry. This ensures that after a rapid swipe or zoom operation, a clear grid interface is ultimately displayed.

[0101] In this embodiment, when a user performs a two-finger zoom or a single-finger swipe on the screen, the electronic device can redraw the grid interface to refresh the grid interface on the screen. Drawing each frame of the grid interface can include two processes: one process is to draw the current grid interface (which includes the grid of the first column) based on the image in the cache pool corresponding to the previously updated image; the other process is to estimate a new cache range based on the current grid interface. When there are newly added images between the current and new cache ranges, the batch decoding data of the newly added images is stored in the cache pool to update the cache pool, thus facilitating the next drawing of the grid interface of the first column.

[0102] For example, taking a single-finger swipe operation scenario as an example, Figure 13 This illustrates the relationship between batch decoding data and the drawing grid interface in each drawing cycle.

[0103] like Figure 13 As shown, during the rendering cycle of the i-th frame grid interface, the electronic device first retrieves the image used to render the i-th frame grid interface from the cache pool and renders it onto the View control, thus obtaining the i-th frame grid interface, and then displays the i-th frame grid interface. Additionally, the electronic device can estimate a new cache range based on the center AP in the i-th frame grid interface, then compare the new cache range with the current cache range in the cache pool to determine the i-th batch decoding data, decode the i-th batch decoding data, and store the i-th batch decoding data into the cache pool after decoding is complete.

[0104] During the rendering cycle of the (i+1)th frame of the grid interface, the electronic device first retrieves the image for rendering the (i+1)th frame of the grid interface from the cache pool and renders it onto the View control, thus obtaining the (i+1)th frame of the grid interface, and then displays the (i+1)th frame of the grid interface. Additionally, the electronic device can estimate a new cache range based on the center AP in the (i+1)th frame of the grid interface, then compare the new cache range with the current cache range in the cache pool to determine the (i+1)th batch decoding data, decode the (i+1)th batch decoding data, and store the (i+1)th batch decoding data into the cache pool after decoding is complete.

[0105] During the rendering cycle of the (i+2)th frame of the grid interface, the electronic device first retrieves the image for rendering the (i+2)th frame of the grid interface from the cache pool and renders it onto the View control, thus obtaining the (i+2)th frame of the grid interface, and then displays the (i+2)th frame of the grid interface. Additionally, the electronic device can estimate a new cache range based on the center AP in the (i+2)th frame of the grid interface, compare the new cache range with the current cache range in the cache pool, determine the (i+2)th batch decoding data, decode the (i+2)th batch decoding data, and store the (i+2)th batch decoding data into the cache pool after decoding is complete.

[0106] In the above scheme, when the user performs a single-finger swipe operation on the screen, the electronic device can estimate a new cache range based on the current grid interface. When there are new images in the current cache range and the new cache range, the batch decoding data of the new images is stored in the cache pool. This makes it easier to directly read the already decoded images from the cache pool when drawing the grid interface in the next frame, thereby reducing the probability that some grid images cannot be successfully decoded due to the image decoding speed not being able to keep up with the user's operation speed.

[0107] In this embodiment of the application, when a user opens a photo album in the gallery application, the gallery application will execute the initialization process corresponding to that album and display the default grid interface (i.e. the first frame grid interface).

[0108] For example, Figure 14 A diagram illustrating the initialization process corresponding to the photo album is provided. For example... Figure 14 As shown, the method may include the following steps S1 to S10. The executing entity of the method may be, for example, Figure 11 The functional modules shown.

[0109] S1, the touch driver samples the user's touch operation according to the period T.

[0110] The aforementioned cycle, also known as the deployment cycle or reporting cycle, is the cycle for the touch driver to collect user touch operations.

[0111] Taking a touch driver that collects data 240 times per second as an example, the period T = 1 / 240 = 4.16 ms, meaning the touch driver collects touch data every 4.16 ms. If, during a certain period, the user presses the touchscreen with one or two fingers, the touch driver can determine the touch position of the single or two fingers on the display screen based on voltage changes, etc.

[0112] S2, the touch driver receives the user's click operation on the gallery application icon, launches the gallery application, and displays the album card page.

[0113] For example, when a user wants to view pictures or videos in a gallery app, the user can click on such... Figure 1 The gallery icon 01 is shown in (a) above. The touch driver receives the user's click on the gallery icon 01 and passes the click operation to the application processor. The application processor launches the gallery application corresponding to the gallery icon 01. After launching the gallery application, the gallery application displays as shown below. Figure 1 The card page shown in (b) is an example. The card page provides album cards for the camera, all photos, and videos; therefore, it is also called an album card page. Each album card serves as an entry point to an album.

[0114] It should be noted that the gallery application typically displays album cards by default. If the user clicks on... Figure 1 If the photos, memories, or creations in the bottom bar shown in (b) are displayed, the gallery app will switch to another page.

[0115] S3, the touch driver receives the user's action of clicking on a card in the album card page and notifies the gallery application.

[0116] S4, the gallery application triggers the data loader initialization.

[0117] S5, the data loader executes the initialization process and loads the first batch of data.

[0118] The initialization process of the aforementioned data loader may include: loading the album's data description (e.g., which folder the album is in), loading the names of the images in the album, loading the dimensions of the images in the album, and loading the exposure of the images, etc. The first batch of data mentioned above consists of images from the grid interface that is displayed by default after opening the album.

[0119] In addition, the gallery application can load and initialize low-density and high-density views. Low-density view initialization includes initializing the number of columns in the two day views and initializing the size of the low-density view. High-density view initialization includes initializing the number of columns in the month and year views and initializing the size of the high-density view.

[0120] S6, the data loader triggers the image loader initialization.

[0121] S7, the image loader creates 3 pending tasks (such as...) Figure 12 The suspended tasks are shown as 1, 2, and 3.

[0122] The three suspended tasks mentioned above are used for batch decoding of 512-pixel, 256-pixel, and 96-pixel images, respectively. Each of these tasks is initially set to a suspended state upon creation. They only become active when a task list containing a new cache range is received. This is understandable, as a new cache range typically includes multiple images corresponding to a specific cache size, allowing the suspended tasks to be used for batch image decoding.

[0123] The aforementioned 512-size image refers to an image with a width of 512 pixels and a height of 512 pixels. A 512-size image can be used to create a grid interface in level 3. A 256-size image refers to an image with a width of 256 pixels and a height of 256 pixels. A 256-size image can be used to create a grid interface in level 5. A 96-size image refers to an image with a width of 96 pixels and a height of 96 pixels. A 96-size image can be used to create a grid interface in level 13.

[0124] It should be noted that the 512-pixel, 256-pixel, and 96-pixel images, as well as the correspondence between each image size and the corresponding tier, are merely illustrative examples and do not limit the scope of this application. Furthermore, referring to the description of the above embodiments, the image library application can also set 31 tiers. Tier 31 can correspond to smaller images (e.g., 48 pixels wide * 48 pixels high). The decoding method for images corresponding to tier 31 is the same as the decoding method for images corresponding to other tiers, or other decoding methods can be used; this application does not limit this approach.

[0125] S8, the image loader creates one suspended task (e.g. Figure 12 The suspended task is shown in Figure 4.

[0126] The aforementioned suspended task is initially set to a suspended state upon creation. It only switches from a suspended state to an active state when it receives a decoding request for a single image in the queue. It's understandable that since decoding of 512-pixel, 256-pixel, and 96-pixel images can all fail, a "decoding request for a single image" can be for any of these sizes.

[0127] S9, the image loader creates 3 cache pools (e.g. Figure 9 The 512 cache pool, 256 cache pool, and 96 cache pool shown are used to store images of size 512, 256, and 96, respectively.

[0128] In some embodiments, the cache size of each of the three cache pools is R times the number of images displayed on the screen at the corresponding level, where R is an integer greater than or equal to 2.

[0129] For example, taking R=2 as an example. If at level 3, the screen displays 24 grids (3 columns * 8 rows), then the 512 buffer pool is used to store 48 images of size 512. If at level 5, the screen displays 75 grids (5 columns * 15 rows), then the 256 buffer pool is used to store 150 images of size 256. If at level 13, the screen displays 455 grids (13 columns * 35 rows), then the 96 buffer pool is used to store 910 images of size 96.

[0130] In some embodiments, the above three cache pools are least recently used (LRU) cache pools.

[0131] S10, the gallery application draws and displays the first frame of the grid interface.

[0132] For example, the first frame of the grid interface can be like this: Figure 1The grid interface shown in (c) is shown in the image.

[0133] It should be noted that the images displayed in the first grid of the photo album are pre-set. For example, the first grid image in the first grid is a thumbnail of the earliest original image or video captured from the album, and the last grid image is a thumbnail of the most recently captured original image or video from the album. "Capture time" refers to the time when the original image or video was saved to the album. This original image or video may have been taken with a camera, downloaded from a network device, or obtained through a screenshot.

[0134] For the specific implementation of drawing the first frame of the grid interface in the gallery application, please refer to the following embodiment for the drawing method of the i-th frame grid interface, which will not be elaborated here.

[0135] In the initialization process described above, when a user opens a photo album in the Gallery app, the Gallery app executes the initialization process corresponding to that album, creating a suspended thread for batch decoding, a suspended thread for single decoding, and cache pools for images of different sizes. This allows the Gallery app to directly call these threads to perform the corresponding decoding tasks when the user performs swiping or zooming operations on the screen.

[0136] Referring to the description of the above embodiments, when a user performs a zoom or swipe operation on the screen, the electronic device can execute the decoding scheme provided in the embodiments of this application. For ease of understanding, the decoding process in these two scenarios will be described in detail below through Embodiment 1 and Embodiment 2.

[0137] Example 1 In a swipe operation scenario, when an electronic device receives a swipe from the user, it triggers the rendering of the grid interface. Since the grid interface only scrolls vertically, the number of columns in each grid interface is equal before and after the screen refresh, and the two grid interfaces use the same decoded data from the cache pool. During the rendering of the current grid interface, it can be rendered based on the image in the cache pool updated by the previous swipe operation.

[0138] For example, Figure 15 This demonstrates the overall process of drawing a grid interface in a sliding operation scenario.

[0139] like Figure 15As shown, after the electronic device draws the (i-1)th frame grid interface, it can estimate a new buffer range based on the center AP of the (i-1)th frame grid interface, compare the new buffer range with the current buffer range in the buffer pool, determine the (i-1)th batch decoding data, decode the (i-1)th batch decoding data, and store the (i-1)th batch decoding data in the buffer pool after decoding. During the drawing cycle of the i-th frame grid interface, the electronic device can execute the following S11 to S20.

[0140] S11, the touch driver continues to sample the user's touch operations according to period T.

[0141] S12, when the grid interface of the (i-1)th frame is displayed, the touch driver receives the user's single-finger swipe operation and reports the swipe distance dy to the gallery application.

[0142] The single-finger swipe operation described above can be either a single-finger swipe along a first direction or a single-finger swipe along a second direction. The first direction is from the bottom of the screen to the top of the screen. The second direction is the opposite of the first direction, from the top of the screen to the bottom of the screen.

[0143] S13, the gallery app slides the View control a distance dy along a first or second direction.

[0144] S14, the gallery application triggers page / interface rendering.

[0145] When the gallery application triggers page / interface rendering, the first and last grid cells in the display area of ​​the screen (i.e., the updated grid interface) can be determined based on the sliding distance dy of the View control.

[0146] As an example, the View control can update the AP value of each cell according to the formula: AP + s * g. Here, s represents the number of rows the View control moves, and g represents the current position of the View control (i.e., the number of cells / columns in the drawing area).

[0147] For each grid cell in the grid interface, you can execute S15 to S18 below to obtain the image corresponding to each grid cell.

[0148] S15, the gallery application requests the data loader to obtain the bitmap wrapper corresponding to a cell.

[0149] The aforementioned "bitmap encapsulation" refers to encapsulating the bitmap in a Hierachy pool. The reason for using Hierachy encapsulation in this application embodiment is to switch the sharpness of the bitmap. For example, if the sliding operation in S12 is received when displaying a grid interface with 3 levels, then S15 requests to obtain a 512-size bitmap corresponding to a grid. However, if there is no 512-size bitmap in the 512 cache pool, but there is a 256-size bitmap corresponding to this grid in the 256 cache pool, then the 256-size bitmap can be returned.

[0150] S16, the data loader sends a request to the image loader to retrieve the previously cached bitmap from the cache pool.

[0151] "The bitmap cached last time" refers to the bitmap in the cache pool after the cache pool is updated during the drawing cycle of the (i-1)th frame of the grid interface. This may include the batch decoding data of the (i-1)th time, and may also include the decoding data cached before the drawing cycle of the (i-1)th frame of the grid interface.

[0152] S17, the image loader prioritizes retrieving the bitmap corresponding to this grid from the cache pool that meets the usage size, and prioritizes returning the bitmap of the corresponding size to the data loader.

[0153] Referring to the description of S9 in the above embodiment, the image loader creates three cache pools: a 512 cache pool, a 256 cache pool, and a 96 cache pool. The 512 cache pool is used to store images of size 512, the 256 cache pool is used to store images of size 256, and the 96 cache pool is used to store images of size 96. The image loader can set priorities for these three cache pools; for example, the 512 cache pool has the highest priority, the 256 cache pool has the second highest priority, and the 96 cache pool has the third highest priority. The first priority is higher than the second priority, and the second priority is higher than the third priority.

[0154] The image loader can determine the size of the image to be drawn based on the size of the grid in the (i-1)th frame (i.e., the current grid position), and then determine the cache pool corresponding to the image to be drawn.

[0155] Taking the (i-1)th frame of the grid interface, which includes 3 grids, as an example, the size of the image to be drawn corresponding to one grid is 512 pixels wide * 512 pixels high, and the corresponding cache pool is a 512-pixel cache pool. If the 512-pixel cache pool has the decoding data corresponding to this grid, then execute S18 as follows. If the 512-pixel cache pool does not have the decoding data corresponding to this grid, then check if the second priority 256-pixel cache pool has the decoding data corresponding to this grid. If it does, then execute S18 as follows. If the 256-pixel cache pool also does not have the decoding data corresponding to this grid, then check if the third priority 96-pixel cache pool has the decoding data corresponding to this grid. If it does, then execute S18 as follows. It can be understood that when a high-priority cache pool has the decoding data corresponding to a grid, the higher-sized decoding data is returned first; when a high-priority cache pool does not have the decoding data corresponding to a grid, the lower-sized decoding data is returned. In this way, the decoding data corresponding to each grid can be obtained.

[0156] For the specific implementation of S17, please refer to the following embodiments. Figure 22 The relevant descriptions are not elaborated here.

[0157] S18, the data loader returns the successfully acquired bitmap wrapper to the gallery application.

[0158] S19, after obtaining the bitmap encapsulation corresponding to each grid, the gallery application draws the i-th frame grid interface based on the moved View control and the decapsulated bitmap data, and sends the i-th frame grid interface to the display.

[0159] Specifically, the image library application can draw each decapsulated bitmap into its corresponding grid, thus obtaining the i-th frame grid interface, and then display the i-th frame grid interface. If the (i-1)-th frame grid interface is as follows... Figure 3 If the grid interface is shown in (a) above, then the grid interface of the i-th frame can be as follows: Figure 3 The grid interface shown in (b) can be as follows: The grid interface of the (i+1)th frame can be... Figure 3 The grid interface shown in (c) is shown in the image.

[0160] S20, based on the first and last grid cells in the display area after the View control has slid a distance dy, the gallery application obtains the batch decoding data for the i-th time and caches it in the cache pool corresponding to the current gear.

[0161] It should be noted that after the View control is slid a distance dy along the first or second direction in S13, the gallery application can already calculate the first and last grid cells in the display area. Therefore, this embodiment does not specifically limit the execution order of S14-S19 and S20. The first and last grid cells in the display area after the View control has slid a distance dy are the first and last grid cells in the grid boundary of the i-th frame.

[0162] Specifically, the gallery application can estimate a new cache range based on the first and last grid cells in the display area after the View control has slid a distance dy. Then, it compares the new cache range with the current cache range in the cache pool to determine the i-th batch decoding data, decodes the i-th batch decoding data, and stores the i-th batch decoding data into the cache pool after decoding is complete.

[0163] For the specific implementation of S20, please refer to the following embodiments. Figure 16 The relevant descriptions are not elaborated here.

[0164] If, while displaying the i-th frame of the grid interface, the user continues to swipe, the rendering cycle for the (i+1)-th frame of the grid interface begins. During this rendering cycle, the gallery application first retrieves the image from the cache pool to render the (i+1)-th frame of the grid interface and renders it onto the moved View control, thus obtaining the (i+1)-th frame of the grid interface. This (i+1)-th frame of the grid interface is then displayed. Additionally, the gallery application can estimate a new cache range based on the center AP in the (i+1)-th frame of the grid interface. It then compares this new cache range with the current cache range in the cache pool to determine the (i+1)-th batch decoding data. This batch decoding data is then decoded, and after decoding, it is stored in the cache pool.

[0165] In the aforementioned swipe operation scenario, when the electronic device receives the user's swipe action, it triggers the rendering of the grid interface and the updating of images in the cache pool. Since the images that may be used in the next frame of the grid interface are pre-stored in the cache pool, the decoded data can be directly retrieved from the cache pool when rendering the next frame of the grid interface. This reduces the probability that some grid images may fail to be decoded successfully because the image decoding speed may not be able to keep up with the user's operation speed.

[0166] Based on the description of the sliding operation scenario in the above embodiments, during the rendering cycle of each grid interface frame, the electronic device estimates the image that may be used in the next grid interface frame based on the currently rendered grid interface, and pre-decodes and stores it in the cache pool. The following, in conjunction with... Figure 16 This document provides an example illustrating the process of updating the cache pool during a sliding operation.

[0167] like Figure 16 As shown, the method may include the following steps S21 to S30.

[0168] S21, the gallery application calculates the center AP in the first and last grids of the display area based on the sliding distance dy of the View control.

[0169] If the access point (AP) in the first cell of the display area is represented as windowStartAP, and the AP in the last cell of the display area is represented as windowEndAP, then the center AP can be represented as: centerAP=(windowStartAP+windowEndAP) / 2.

[0170] For example, if windowStartAP=10 and windowEndAP=30, then centerAP=20.

[0171] S22, the gallery application retrieves the grid size for the current level.

[0172] Taking a screen width of 1200 pixels as an example, at level 3, the size of each grid is 400 pixels wide and 400 pixels high; at level 5, the size of each grid is 240 pixels wide and 240 pixels high; and at level 13, the size of each grid is 93 pixels wide and 93 pixels high.

[0173] S23, the gallery application sends the central AP to the image loader through the data loader.

[0174] S24, the gallery application sends the current grid size to the image loader via the data loader.

[0175] The execution order of S23 and S24 is not limited in this embodiment; for example, they can be executed simultaneously or sequentially.

[0176] S25, the image loader determines the new cache range based on the center AP and the grid size.

[0177] After obtaining the grid size, the image loader can calculate the total number of grids in the screen display area at the current resolution. Then, the image loader performs the following operations: subtracting the total number of grids from the center AP to obtain the AP of the first image in the new cache range; and / or adding the total number of grids to the center AP to obtain the AP of the last image in the new cache range.

[0178] For example, at speed 3, the screen displays 24 grids (3 columns * 8 rows); at speed 5, the screen displays 75 grids (5 columns * 15 rows); and at speed 13, the screen displays 455 grids (13 columns * 35 rows).

[0179] If the grid size of the current level is 400 pixels wide and 400 pixels high, then the image loader can determine that the total number of grids corresponding to level 3 is 24, and thus calculate the new cache range as [centerAP-24, centerAP+24].

[0180] If the grid size of the current level is 240 pixels wide and 240 pixels high, then the image loader can determine that the total number of grids corresponding to level 5 is 75, and thus calculate the new cache range as [centerAP-75, centerAP+75].

[0181] If the current grid size is 93 pixels wide and 93 pixels high, the image loader can determine that the total number of grids corresponding to 13 is 455, and thus calculate the new cache range as [centerAP-455, centerAP+455].

[0182] S26, The image loader compares the new cache range with the current cache range to determine the cache range to be updated.

[0183] Referring to the above embodiments Figure 7 and Figure 8 The description states that the new cache range and the current cache range may or may not overlap. When the new cache range and the current cache range overlap, the image loader needs to clear images outside the new cache range and store them in the cache range that needs to be updated (i.e., update the cache range). When the new cache range and the current cache range do not overlap, the image loader needs to clear all images in the current cache range and store all images from the new cache range in the cache pool (i.e., update the cache range).

[0184] For example, Figure 17 This illustrates an example of updating the cache in a downward scrolling scene. Figure 17 As shown, the current cache range is [currentStartAP, currentEndAP], and the new cache range is [updateStartAP, centerAP]. Here, updateStartAP equals centerAP minus the total number of grids corresponding to level 3. The image loader compares the new cache range with the current cache range and determines that the updated cache range is [updateStartAP, updateEndAP].

[0185] For example, Figure 18 This illustrates an example of updating the cache in an upward swipe scenario. Figure 18As shown, the current cache range is [currentStartAP, currentEndAP], and the new cache range is [centerAP, updateStartAP]. Here, updateStartAP equals centerAP plus the total number of grids corresponding to level 3. The image loader compares the new cache range with the current cache range and determines that the updated cache range is [updateStartAP, updateEndAP].

[0186] S27, the image loader initiates batch decoding of the updated cache range.

[0187] S28, the image loader marks the new cache range as a historical range.

[0188] It is understandable that marking the new cache range as the historical range makes it easier to execute S26 above in the next grid drawing cycle.

[0189] S29, The image loader clears cached data that is outside the new cache range.

[0190] After decoding is complete, the image loader caches the decoded data in the cache pool corresponding to the current gear.

[0191] Referring to the above embodiments Figure 12 The description states that if a new image is added, a Fresco task is created for the suspendable task corresponding to the current gear, and this task is submitted to the Fresco submission module. The Fresco submission module submits a decoding request for the new image to the Fresco open-source code. After the Fresco open-source code completes the decoding, it returns the decoded data to the Fresco submission module. The Fresco submission module then submits the decoded data to the cache pool corresponding to the suspendable task and clears cached data that is outside the new cache range.

[0192] Example 2 In zoom operation scenarios, when an electronic device receives a zoom operation from the user, it triggers the rendering of the grid interface. Since the electronic device changes the number of columns in the grid interface during zooming, the current grid interface (including the grid with the first column number) can be drawn based on the image in the cache pool corresponding to the first column number from the last update.

[0193] For example, Figure 19 This demonstrates the overall process of drawing a grid interface in a zoom operation scenario.

[0194] like Figure 19As shown, after the electronic device draws the (i-1)th frame grid interface, it can estimate the next level (first column number) and a new buffer range corresponding to the next level based on the center AP of the (i-1)th frame grid interface. Then, it compares the new buffer range with the current buffer range in the buffer pool corresponding to the next level to determine the (i-1)th batch decoding data. The (i-1)th batch decoding data is then decoded, and after decoding, it is stored in the buffer pool corresponding to the next level (first column number). During the drawing cycle of the i-th frame grid interface (which includes the grid of the first column number), the electronic device can execute the following steps S31 to S40.

[0195] S31, the touch driver continues to sample the user's touch operations according to period T.

[0196] S32, when displaying the (i-1)th frame grid interface, the touch driver receives the user's two-finger zoom operation and reports the two-finger coordinates to the gallery application.

[0197] The above two-finger zoom operation can be used to decrease the distance between two fingers or to increase the distance between two fingers.

[0198] S33, the library application calculates the center of the two fingers and the scaling factor Si based on the two-finger coordinates.

[0199] For example, the touch driver can first receive the user's two-finger press operation, and then receive the user's operation of reducing the distance between the two fingers.

[0200] When the touch driver receives a two-finger press operation from the user, it can transmit the coordinates (x11, y11) and (x12, y12) of the two-finger press to the zoom gesture detector. Figure 19 (Not shown). The zoom gesture detector calculates the coordinates (mBeginFocusx, mBeginFocusy) ​​of the center of the two fingers and the distance d1 between the two fingers based on the coordinates (x11, y11) and (x12, y12).

[0201] The coordinates of the center of the two fingers (mBeginFocusx, mBeginFocusy) ​​can be obtained through the following relationship (1): (1).

[0202] The distance d1 between the two fingers can be obtained by the following relationship (2): (2).

[0203] Taking the coordinates when two fingers are pressed (x11,y11)=(200,200) and coordinates (x12,y12)=(800,1000) as an example. After the zoom gesture detector calculates the two-finger distance d1=1000 pixels, it can be determined that the two-finger distance d1=1000 is greater than the threshold D=540 pixels, the current operation is determined to be valid, and the two-finger center coordinates (500,600) and the two-finger distance d1=1000 pixels are recorded.

[0204] Then, the touch driver receives the user's operation of reducing the two-finger distance, and transmits the coordinates (xi1, yi1) and coordinates (xi2, yi2) after the two-finger moving operation to the zoom gesture detector. i is an integer greater than or equal to 2. i represents the i-th acquisition of the two-finger operation. The zoom gesture detector calculates the two-finger distance di according to the coordinates (xi1, yi1) and the coordinates (xi2, yi2).

[0205] Wherein, the two-finger distance di can be obtained through the following relational expression (3): (3).

[0206] Taking the coordinates (xi1, yi1)=(100,100) and coordinates (xi2, yi2)=(460,580) after the two-finger moving operation as an example. After the zoom gesture detector calculates the two-finger distance di=600 pixels, it can be determined that the two-finger distance di=600 is greater than the threshold D=540 pixels, the current operation is determined to be valid, and the zoom factor Si is calculated according to the two-finger distance di and the two-finger distance d1.

[0207] Wherein, the zoom factor Si can be obtained through the following relational expression (4): (4).

[0208] scale1 is the zoom factor when two fingers are pressed. di is the two-finger distance corresponding to the i-th zoom operation, and d1 is the two-finger distance corresponding to the two-finger pressing operation. It should be understood that if the user reduces the distance between the two fingers, then di<d1; if the user increases the distance between the two fingers, then di>d1; if the user's two fingers do not move, the zoom factor is equal to 1, then the operation is determined to be invalid, and it is not necessary to determine whether to set the two-finger center. In one cycle T, the faster the two-finger moving speed, the larger the difference between di and d1, and the faster the gear switching.

[0209] As an example, multiple zoom levels can be preset in a mobile phone, and a switching threshold is set between any two adjacent zoom levels. Two adjacent zoom levels refer to two zoom levels in which no other zoom levels are set. If the scaling factor of the smaller zoom level is represented by a1, the scaling factor of the larger zoom level is represented by a2, and the switching threshold is represented by a3, then the switching threshold a3 can be calculated using the following relationship (5): (5).

[0210] Taking the scaling factor of a low-density view at level 3 as 1.0 and at level 5 as 0.6 as an example, as shown in Table 1, according to the above relationship (5), the critical point for switching between levels 3 and 5 can be calculated to be 0.8.

[0211] Table 1

[0212] After the electronic device obtains the scaling factor Si based on the relation (4), it can compare the scaling factor Si with the switching gear threshold of 0.8. If Si ≤ 0.8, then it can be determined that the daytime view switching condition is met, and the following S34 is executed. If Si > 0.8, then it can be determined that the daytime view switching condition is not met, and it is not necessary to switch from the first gear (gear 3) daytime view to the second gear (gear 5) daytime view.

[0213] S34, the gallery application triggers page / interface drawing (including the grid of the first column number) based on the center of the two fingers and the zoom level.

[0214] When the gear shifting conditions are met, the gallery application can determine the row and column of the area to be drawn, as well as the AP value of each cell in the drawing area, based on the center of the two fingers and the zoom level.

[0215] For example, such as Figure 20 As shown, before the scaling operation, the AP of the first cell in the display area was 69, and the AP of the last cell was 92. After the scaling operation, the AP of the first cell in the display area was 64, and the AP of the last cell was 118.

[0216] The AP value corresponding to each cell in the drawing area can be determined by the following (a1) to (a8): (a1) Determine the column number Index1 of the first column of the display area of ​​the two-finger center and the next level (i.e. the first column number) in the View control, and the column number Index2 of the last column of the display area of ​​the next level in the View control.

[0217] The View control can also determine the row number (Line1) of the first row of the display area in the next tier and the row number (Line2) of the last row of the display area in the next tier. Based on Index1, Index2, Line1, and Line2, the AP of the first and last cells in the display area of ​​the next tier can be determined.

[0218] (a2) Determine the LP value (also known as selectLP) of the selected grid cell by two fingers and the AP value (also known as selectAP) of the selected grid cell by two fingers.

[0219] The grid selected by the center of the two fingers refers to the grid where the coordinates of the center of the two fingers are located.

[0220] The following steps can be performed on each cell in the View control: (a3) Determine the number of columns (LineNum) and the value of LP of a cell in the View control.

[0221] The column number lineNum of the cell to be adjusted can be determined according to the following formula: .

[0222] in, This represents the highest gear (e.g., 31 gears).

[0223] (a4) Determine whether Index1≤LineNum≤Index2 is satisfied.

[0224] If Index1≤LineNum≤Index2 is satisfied, then execute (a5).

[0225] If Index1≤LineNum≤Index2 is not satisfied, then execute (a8).

[0226] (a5) Determine whether LP≤selectLP is satisfied.

[0227] If LP≤selectLP is satisfied, then execute (a6).

[0228] If LP≤selectLP is not satisfied, then execute (a7).

[0229] (a6) Determine the AP value of this cell according to the first formula.

[0230] The first formula can be: .

[0231] Where nextShowSpan represents the number of columns to display for the next tier. Representing the maximum gear level (e.g., 31 gears), selectLP with two fingers to select the LP value of the grid cell. This represents the number of rows selected.

[0232] (a7) Determine the AP value of this cell according to the second formula.

[0233] The second formula can be: .

[0234] (a8) Set the AP value to -1.

[0235] In this case, AP=-1 means that the actual adaptation position value is invalid, and at this time, a grid can display a blank image.

[0236] After determining the AP value corresponding to each cell in the drawing area, for each cell in the drawing area, the following steps S35 to S38 can be performed to obtain the image corresponding to each cell.

[0237] S35, the gallery application requests the data loader to obtain the bitmap wrapper corresponding to a cell.

[0238] The aforementioned "bitmap encapsulation" refers to encapsulating a bitmap within a Hierachy pool.

[0239] S36, the data loader sends a request to the image loader to retrieve the previously cached bitmap from the cache pool corresponding to the first column number.

[0240] "The bitmap cached last time" refers to the bitmap in the cache pool after the cache pool is updated during the drawing cycle of the grid interface in frame x. This may include the batch decoding data of frame x, and may also include the decoding data cached before the drawing cycle of the grid interface in frame x.

[0241] S37, the image loader prioritizes retrieving the bitmap corresponding to this grid from the cache pool that meets the usage size, and prioritizes returning the bitmap of the corresponding size to the data loader.

[0242] For the specific implementation of S37, please refer to the following embodiments. Figure 22 The relevant descriptions are not elaborated here.

[0243] S38, the data loader returns the successfully acquired bitmap wrapper to the gallery application.

[0244] S39. After obtaining the bitmap encapsulation corresponding to each grid, the gallery application draws the i-th grid interface based on the scaled View control and sends the i-th grid interface to the display.

[0245] Specifically, the image library application can draw each bitmap onto its corresponding grid, thus obtaining the i-th frame grid interface, and then display the i-th frame grid interface. If the (i-1)-th frame grid interface is as follows... Figure 1 If the grid interface is shown in (d) above, then the grid interface of the i-th frame can be as follows: Figure 1 The grid interface shown in (e) can be as follows: The grid interface of the (i+1)th frame can be... Figure 1 The grid interface shown in (f) is shown in the figure.

[0246] For the specific implementation of S35-S38, please refer to the description of S15-S18 in the above embodiments, which will not be repeated here.

[0247] S40, the gallery application obtains the batch decoding data for the i-th time and caches it in the cache pool corresponding to the second column number.

[0248] The second column number is the column number of the next gear position adjacent to the first column number. For example, if the i-th frame grid interface includes 5 grids, the second column number is 13 columns.

[0249] It should be noted that the execution order of S35-S38 and S40 is not specifically limited in the embodiments of this application.

[0250] For the specific implementation of S40, please refer to the following embodiments. Figure 21 The relevant descriptions are not elaborated here.

[0251] After displaying the i-th grid screen, if a zoom operation is received from the user, the rendering cycle for the (i+1)-th grid screen (second column number) begins. During the (i+1)-th grid screen rendering cycle, the gallery application first retrieves the image for rendering the (i+1)-th grid screen from the cache pool corresponding to the second column number, and renders it onto the zoomed View control, thus obtaining the (i+1)-th grid screen, which is then displayed. Additionally, based on the center AP in the (i+1)-th grid screen, the gallery application can estimate the next tier (third column number) and a new cache range corresponding to the next tier. It then compares the new cache range with the current cache range in the cache pool to determine the (i+1)-th batch decoding data, decodes the (i+1)-th batch decoding data, and stores it in the cache pool corresponding to the third column number after decoding. The third column number is the column number of the tier adjacent to the tier of the second column number.

[0252] In the zoom operation scenario described above, when the electronic device receives a zoom operation from the user, it triggers the rendering of the grid interface and the updating of the images in the cache pool. Since the images that may be used in the next frame of the grid interface (including the first column number) are pre-stored in the cache pool corresponding to the first column number, when rendering the next frame of the grid interface, the decoded data can be directly obtained from the cache pool corresponding to the first column number. This reduces the probability that some grid images may fail to be decoded successfully because the image decoding speed may not be able to keep up with the user's operation speed.

[0253] Based on the description of the scaling operation scenario in the above embodiments, during the rendering cycle of each grid interface, the electronic device estimates the next level and the corresponding cache range (i.e., the image that may be used when rendering the grid interface of the next level) based on the currently rendered grid interface, and pre-decodes and stores it in the cache pool. The following, combined with... Figure 21 This document provides an example illustrating the process of updating the cache pool during scaling operations.

[0254] like Figure 21 As shown, the method may include the following steps S41 to S48.

[0255] S41, the gallery application sends the central AP to the image loader through the data loader.

[0256] S42, the gallery app sends the next grid size to the image loader via the data loader.

[0257] The execution order of S41 and S42 is not limited in this embodiment; for example, they can be executed simultaneously or sequentially.

[0258] S43, the image loader determines the new cache range based on the center AP and the grid size.

[0259] After obtaining the grid size, the image loader can calculate the total number of grids in the screen display area for the next tier. Then, the image loader performs the following operations: subtracting the total number of grids from the center AP to obtain the AP of the first image in the new cache range; and / or adding the total number of grids to the center AP to obtain the AP of the last image in the new cache range.

[0260] For example, at speed 3, the screen displays 24 grids (3 columns * 8 rows); at speed 5, the screen displays 75 grids (5 columns * 15 rows); and at speed 13, the screen displays 455 grids (13 columns * 35 rows).

[0261] If the current grid size is 400 pixels wide and 400 pixels high, the image loader can determine that the total number of grids corresponding to the next level (level 5) is 75, and thus calculate the new cache range as [centerAP-75, centerAP+75].

[0262] If the current grid size is 240 pixels wide and 240 pixels high, the image loader can determine that the total number of grids corresponding to the next level (level 13) is 455, and thus calculate the new cache range as [centerAP-455, centerAP+455].

[0263] S44, the image loader compares the new cache range with the current cache range to determine the cache range to be updated.

[0264] S45, the image loader initiates batch decoding of the updated cache range.

[0265] S46, the image loader marks the new cache range as a historical range.

[0266] S47, the image loader clears cached data that is outside the new cache range.

[0267] S48. After decoding is complete, the image loader caches the decoded data in the corresponding cache pool.

[0268] For the specific implementation of S44-S48, please refer to the description of S26-S30 in the above embodiments, which will not be repeated here.

[0269] The above embodiments one and two illustrate that the image loader can preferentially retrieve the bitmap corresponding to this grid from a cache pool that meets the usage size. The following, in conjunction with... Figure 22 The specific implementation method is illustrated with examples.

[0270] like Figure 22 As shown in (a), when each grid in the i-th frame of the grid interface is used to display a bitmap of size 96, the following judgment process is performed for each grid: Determine whether a bitmap of size 96 is cached in the 96-buffer pool. If a bitmap of size 96 is cached in the 96-buffer pool, then return the bitmap of size 96. If a bitmap of size 96 is not cached in the 96-buffer pool, then decode a bitmap of size 48 and return the bitmap of size 48. The bitmap of size 48 can be stored in other memory, such as a data block (DB).

[0271] like Figure 22As shown in (b), when each grid in the i-th frame of the grid interface is used to display a bitmap of size 256, the following judgment process is performed for each grid: Check if a bitmap of size 256 is cached in the 256 cache pool. If a bitmap of size 256 is cached in the 256 cache pool, then return the bitmap of size 256. If a bitmap of size 256 is not cached in the 256 cache pool, then check if a bitmap of size 96 is cached in the 96 cache pool. If a bitmap of size 96 is cached in the 96 cache pool, then return the bitmap of size 96. If a bitmap of size 96 is not cached in the 96 cache pool, then decode a bitmap of size 48 and return the bitmap of size 48.

[0272] like Figure 22 As shown in (c), when each grid in the i-th frame of the grid interface is used to display a bitmap of size 512, the following judgment process is performed for each grid: Check if a bitmap of size 512 is cached in the 512 cache pool. If a bitmap of size 512 is cached in the 512 cache pool, then return the bitmap of size 512. If a bitmap of size 512 is not cached in the 512 cache pool, then check if a bitmap of size 256 is cached in the 256 cache pool. If a bitmap of size 256 is cached in the 256 cache pool, then return the bitmap of size 256. If a bitmap of size 256 is not cached in the 256 cache pool, then check if a bitmap of size 96 is cached in the 96 cache pool. If a bitmap of size 96 is cached in the 96 cache pool, then return the bitmap of size 96. If there is no bitmap with size 96 in the 96 cache pool, then decode the bitmap with size 48 and return the bitmap with size 48.

[0273] In the above solution, prioritizing the retrieval of bitmaps from a cache pool that meets the required size ensures image clarity. When a bitmap cannot be retrieved from a cache pool that meets the required size, retrieving a lower-sized bitmap avoids blank areas in the grid interface. In scenarios involving rapid scrolling or zooming, due to the high refresh rate of the grid interface, the use of lower-sized bitmaps in some grid cells is generally imperceptible to the user.

[0274] After a rapid swipe or zoom operation, electronic devices often remain on the last grid screen for an extended period. In such cases, if this grid screen uses a low-sized bitmap, it may be noticeable to the user. To address this issue, this application also provides a method for decoding a single image within the last grid screen.

[0275] For example, Figure 23 A decoding method for a single image in the last frame of a grid interface is shown.

[0276] like Figure 23 As shown, the i-th frame grid interface is taken as the last frame grid interface.

[0277] The method may include the following steps S49 to S55.

[0278] S49, the touch driver receives the user's hand-raise action and notifies the gallery app. The gallery app then notifies the image loader.

[0279] S50, the image loader determines whether the bitmap of each cell in the i-th frame grid interface is smaller than the target size.

[0280] The i-th frame of the grid interface mentioned above is the last frame of the grid interface displayed when you raise your hand.

[0281] If the i-th frame grid interface includes 3 columns of grids, then the target size is 512.

[0282] If the i-th frame grid interface includes 5 columns, then the target size is 256.

[0283] If the i-th frame grid interface includes 13 columns, then the target size is 96.

[0284] When the bitmap of a certain cell is smaller than the target size, the image loader executes the following S51.

[0285] S51, the image loader initiates a decoding task for the single bitmap of this grid.

[0286] S52, the image loader obtains the decoded data of a single bitmap for this grid.

[0287] like Figure 12 As shown, for a specific grid, when the electronic device tries to retrieve an image from the cache pool using the `get()` interface, it fails to do so. In this case, the `get()` interface submits a single decoding task for the unretrieved image to a single decoding module, marking it as a callback. The single decoding module's suspended task switches from a suspended state to an active state, creates a Fresco task, and submits the task to the Fresco submission module. The Fresco submission module submits a decoding request for the unretrieved image to the Fresco open-source code. After the Fresco open-source code completes decoding, it returns the decoded data of the single image to the Fresco submission module. Since the decoded data of this image has been marked as a callback, the Fresco submission module can return the decoded data to the `get()` interface via an asynchronous sub-thread callback function.

[0288] S53, the image loader triggers the View control to redraw the grid interface.

[0289] S54, the gallery application redraws the grid interface of the i-th frame (i.e., draws the grid interface of the (i+1)-th frame).

[0290] S55, the gallery application will send the redrawn i-th frame grid interface for display.

[0291] In some embodiments, if multiple grid cells in the i-th frame of the grid interface are smaller than the target size, then according to Android's native logic, multiple redraw operations will be performed. For example... Figure 24 As shown, in the i-th frame of the grid interface, the grid in the third row and third column and the grid in the fifth row and first column are smaller than 512. The electronic device first redraws the grid in the third row and third column to obtain the (i+1)-th frame of the grid interface, and then redraws the grid in the fifth row and first column to obtain the (i+2)-th frame of the grid interface. It is understandable that, due to the high screen refresh rate of electronic devices, it is difficult for the human eye to perceive the addition of one or two grid frames.

[0292] In the above solution, by redrawing the last grid interface, the bitmap size of each grid in the last grid interface can be made higher, so that the user perceives the grid interface as clearer after scrolling or zooming.

[0293] For example, Figure 25 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application.

[0294] like Figure 25 As shown, the electronic device 100 may include a processor 110, internal memory 120, buttons 130, a sensor module 140, a display screen 150, an audio module 160, a speaker 160A, a receiver 160B, a microphone 160C, a headphone jack 160D, etc. The sensor module may include a touch sensor.

[0295] The processor 110 can be used to execute the scrolling display method in the above embodiments. The processor 110 may include one or more processing units, such as a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), an image signal processor (ISP), etc.

[0296] Internal memory 120 can be used to store executable program code, including instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 120. Internal memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application (APP) required for a function, such as a gallery application. The data storage area may store configuration files for each APP, as well as data created during the use of electronic device 100.

[0297] The display screen 150 includes a display panel for displaying various images in a gallery application.

[0298] A touch sensor, also known as a "touch panel," can be located on the display screen 150. The touch sensor and the display screen 150 together form a touchscreen, also called a "touch display." The touch sensor can detect touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event and provide visual output related to the touch operation through the display screen 150. In some embodiments, the touch sensor may also be located on the surface of the electronic device 100, in a different position than the display screen 150.

[0299] This application also provides a computer-readable storage medium storing computer instructions. When the computer-readable storage medium is operated on an electronic device, it causes the electronic device to perform the methods shown in the above embodiments. The computer instructions can be stored in the computer-readable storage medium or transferred from one computer-readable storage medium to another. The computer-readable storage medium can be any usable medium that a computer can access, or it can include one or more data storage devices such as servers or data centers that can be integrated with the medium. The usable medium can be a magnetic medium, an optical medium, or a semiconductor medium, etc.

[0300] This application also provides a computer program product, which includes computer program code. When the computer program code is run on an electronic device, it causes the electronic device to perform the methods described in the above embodiments.

[0301] The electronic devices, computer-readable storage media, and computer program products provided in the embodiments of this application are all used to execute the methods provided in the above embodiments. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects corresponding to the methods provided above, and will not be repeated here.

[0302] In the description of this application, " / " means "or". For example, A / B can mean A or B. In the description of this application, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0303] In the specification and claims of this application, the terms "first" and "second," etc., are used to distinguish different objects or to distinguish different treatments of the same object, rather than to describe a specific order of the objects. For example, "first operation" and "second operation," etc., are used to distinguish different operations, rather than to describe a specific order of operations. In the embodiments of this application, "multiple" refers to two or more.

[0304] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0305] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A decoding method based on a grid interface, characterized in that, The method includes: Receive the user's first operation, which is a zoom operation; In response to the first operation, a first grid interface is displayed; the coordinates of the center of the two fingers are used to display the image in the selected center grid of the first grid interface, and the total number of grids of the grid interface to be displayed on the screen is subtracted from and added to obtain a first cache range; based on the first cache range, a first batch of decoded images are obtained; the first batch of decoded images are stored in a cache pool corresponding to the size of the first grid; wherein, the first grid interface includes a grid with a first number of columns, the total number of grids of the grid interface to be displayed on the screen is determined according to a second number of columns, the second number of columns is the number of columns of the next level adjacent to the level of the first number of columns, the number of the first batch of decoded images is greater than the total number of grids of the grid interface to be displayed on the screen, and the size of the first grid is determined according to the second number of columns; Receive the user's second operation, which is a zoom operation; In response to the second operation, for each adaptation position value from the first adaptation position value to the second adaptation position value: if the cache pool corresponding to the first grid size includes a decoded image corresponding to an adaptation position value, then a second grid interface is drawn based on the decoded image corresponding to an adaptation position value in the cache pool corresponding to the first grid size; if the cache pool corresponding to the first grid size does not include a decoded image corresponding to an adaptation position value, then a second grid interface is drawn based on the decoded image corresponding to an adaptation position value in the cache pool corresponding to the second grid size; the second grid interface is displayed; wherein the second grid interface includes grids of the second column number, the resolution of the decoded image stored in the cache pool corresponding to the second grid size is less than the resolution of the decoded image stored in the cache pool corresponding to the first grid size, and the second grid size is determined according to the third column number, the third column number being greater than the second column number; The number of columns in the first column, the second column, and the third column are preset numbers.

2. The method according to claim 1, characterized in that, The coordinates of the center of the two fingers are determined based on the coordinates when the user presses down with two fingers, from the time the two fingers are pressed down until the first operation and the second operation, the user's two fingers have not left the screen.

3. The method according to claim 1, characterized in that, The first grid interface and the second grid interface are drawn based on the first view control, and the size of the first view control is larger than the screen size; The method further includes: Select a grid in the first grid interface based on the coordinates of the center of the two fingers, and determine the drawing area of ​​the second grid interface in the first view control; Based on the coordinates of the center of the two fingers, select the layout position value and adaptation position value of the selected grid in the first grid interface, determine the first adaptation position value and the second adaptation position value, the first adaptation position value is the adaptation position value of the first grid in the drawing area, and the second adaptation position value is the adaptation position value of the last grid in the drawing area.

4. The method according to claim 1, characterized in that, The step of obtaining the first batch of decoded images based on the first cache range includes: Compare the first cache range and the second cache range, where the second cache range is the range of decoded images stored in the cache pool corresponding to the first grid size before receiving the first operation; In the case where there are overlapping decoded images in the first cache range and the second cache range, the decoded images that exceed the first cache range in the cache pool corresponding to the first grid size are cleared, and the non-overlapping decoded images are stored in the cache pool corresponding to the first grid size to obtain the first batch of decoded images; If there are no overlapping decoded images in the first cache range and the second cache range, clear all decoded images of the second cache range in the cache pool corresponding to the first grid size, and store all decoded images of the first cache range in the cache pool corresponding to the first grid size to obtain the first batch of decoded images.

5. The method according to claim 3, characterized in that, The first view control inherits from Android's view control or from the recycle bin view control. The Android view control does not have a table view, while the recycle bin view control does have a table view, and the table view supports responding to user touch operations.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Receive user gestures such as raising their hand; In response to the hand-raising operation, it is determined whether the resolution of each image in the decoded image from the first adaptation position value to the second adaptation position value is less than the first resolution; if the resolution of the first image is less than the first resolution, a decoding task for the first resolution is started to obtain a second image of the first resolution; the second image is used to replace the first image to draw a third grid interface; the third grid interface is displayed; wherein, the image content of the first image and the second image are the same, and the first resolution corresponds to the second column number.

7. The method according to any one of claims 1 to 5, characterized in that, Before receiving the user's first operation, the method further includes: Receive third-party operations from the user on the album cards; In response to the third operation, a first suspended task, N second suspended tasks, and N cache pools are created; The first suspended task is used to process the decoding task of a single image, each of the N second suspended tasks is used to process the decoding task of multiple images of one resolution, and each of the N cache pools is used to store the decoded images of one resolution.

8. An electronic device, characterized in that, The electronic device includes: one or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 7.

9. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being used to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 7.

Images