Method of organizing a gallery and electronic device

Abstract

This application provides a method and electronic device for organizing an image library. The method includes: acquiring information about each image in at least one feature dimension from a plurality of images in the image library, the at least one feature dimension including a semantic feature dimension, and the information about each image in the semantic feature dimension including information obtained by semantic analysis of the content of each image; clustering the plurality of images based on the information about each image in the semantic feature dimension, and generating one or more recommended albums. Through this method and electronic device, the electronic device can automatically organize images in the image library and recommend albums to users. The generated recommended albums combine information about the images in the semantic feature dimension, which can meet users' personalized and fine-grained demands for album categories.

Description

Technical Field

[0001] This application relates to the field of terminal display, and more specifically, to a method and electronic device for organizing a gallery. Background Technology

[0002] As users take photos with their electronic devices, the number of pictures stored on the devices increases, making it difficult for them to quickly find the pictures they want to view. Therefore, automatic image organization is particularly important. Summary of the Invention

[0003] This application provides a method and electronic device for organizing an image library. Through this method and electronic device, the electronic device can automatically organize the images in the image library and recommend albums to users. The generated recommended albums combine information from the semantic feature dimension of the images, which can meet users' personalized and fine-grained demands for album categories.

[0004] In a first aspect, a method for organizing an image library is provided. The method is applied to a first device and includes: acquiring information about each image in a plurality of images in the image library in at least one feature dimension, the at least one feature dimension including a semantic feature dimension, the information about each image in the semantic feature dimension including information obtained by semantic analysis of the content of each image; and clustering the plurality of images based on the information about each image in the at least one feature dimension to generate one or more recommended albums.

[0005] In this embodiment, the electronic device can cluster the images in the gallery based on the information of the semantic feature dimension of the images in the gallery, and generate one or more recommended albums. This can not only automatically organize the images in the gallery, but also meet the user's personalized and fine-grained requirements for album organization, and enable the user to quickly find target images, thereby improving the efficiency of the user in organizing and browsing images.

[0006] In conjunction with the first aspect, in one possible implementation, for any one of the plurality of images, the information of the image in the semantic feature dimension includes one or more of the following: a description of the foreground of the image, a description of the background of the image, a description of the relationship between objects in the image, a description of objects in the image, and an overview of the content of the image.

[0007] In this embodiment, the electronic device can cluster images in the gallery based on information in the semantic feature dimension, and generate one or more recommended albums, which can meet users' personalized and fine-grained demands for album organization.

[0008] In conjunction with the first aspect, in one possible implementation, the description of the object in the image includes one or more of the following: a description of the object's behavior, a description of the object's appearance, a description of the object's expression, and a description of the object's clothing.

[0009] In conjunction with the first aspect, in one possible implementation, the method further includes: constructing a proximity network corresponding to the plurality of images by analyzing the clustering results of the plurality of images, the proximity network being used to indicate the degree of proximity between a first object and one or more other objects and / or the degree of proximity between a second object and one or more other objects, wherein the first object, the second object, and the other objects are objects appearing in at least one of the plurality of images.

[0010] In some embodiments, the first object may be one or more of a person, an animal, or an item; the second object may be one or more of a person, an animal, or an item; and other objects may be one or more of a person, an animal, or an item.

[0011] In this embodiment of the application, an affinity network can be obtained based on the image clustering results. This affinity network can be used, for example, to further classify the clustered album into multiple sub-albums or to further cluster the clustered album into multiple sub-albums. It can also be used to merge the clustered albums to obtain a merged album. Furthermore, it can be used to generate titles for the clustered albums, classified sub-albums, clustered sub-albums, and / or merged albums. It can also serve as a basis for determining the recommendation order of multiple recommended albums and / or for automatic image cleanup.

[0012] In conjunction with the first aspect, in one possible implementation, the first object is the user of the first device.

[0013] In conjunction with the first aspect, in one possible implementation, the method further includes: generating a title for the first recommended album based on information about each image in the first recommended album in the semantic feature dimension.

[0014] Optionally, taking recommended album 1 in one or more recommended albums as an example, the title of recommended album 1 can be generated based on the information of the images belonging to recommended album 1 in the semantic feature dimension.

[0015] In this embodiment, recommended album titles can be generated based on information in the semantic feature dimension of the image, and personalized and interesting album titles can be generated for the album.

[0016] In conjunction with the first aspect, in one possible implementation, the method further includes: generating a title for at least one of the one or more recommended albums based on the intimacy network.

[0017] In some embodiments, taking a first recommended album in one or more recommended albums as an example, in the process of generating the title of the first recommended album, information on the semantic feature dimension of each image in the first recommended album can be used, as well as a proximity network, which includes information for indicating the degree of proximity between multiple objects in the images of the first recommended album.

[0018] In other words, the title of the first recommended album can be generated based on the semantic feature information of each image in the first recommended album; the title of the first recommended album can also be generated based on the intimacy network; and the title of the first recommended album can also be generated based on the semantic feature information of each image in the first recommended album and the intimacy network.

[0019] In conjunction with the first aspect, in one possible implementation, the method further includes: the first device displaying a first interface, the first interface including at least one of the one or more recommended albums.

[0020] In conjunction with the first aspect, in one possible implementation, the at least one recommended album is multiple recommended albums, and the display order of the multiple recommended albums on the first interface is determined according to the image shooting time, or the display order of the at least one recommended album on the first interface is determined according to the user's degree of interest in the event corresponding to the recommended album, wherein the degree of interest is obtained by analyzing the clustering results of the multiple images.

[0021] In some possible implementations, the recommended albums obtained from clustering are displayed at different times, showing the recommended album to the user at the corresponding recommendation time. The recommendation time for a given album may be related to factors such as the current time, the user's location, or currently occurring events.

[0022] In some embodiments, the timing of recommending albums on the first interface can be determined based on the current time. The electronic device can prioritize recommending albums that match the current time to the user. For example, when the electronic device recognizes that the current time is the Dragon Boat Festival, it will prioritize displaying albums related to the Dragon Boat Festival on the interface.

[0023] In some embodiments, the display order of recommended albums on the first interface can be determined based on the user's current location. The electronic device can prioritize recommending albums that match the user's current location. For example, when the electronic device detects that the user has arrived in Huangshan, it will prioritize displaying albums related to Huangshan on the interface.

[0024] In some embodiments, the display order of recommended albums on the first interface can be determined based on the current event. The electronic device can prioritize recommending albums that match the current event to the user. For example, when the electronic device detects that the user is cycling through a sports and health program, or detects that the user has taken a photo of cycling through the electronic device, albums related to the cycling event are displayed on the interface first.

[0025] In this embodiment of the application, the generated recommended albums can be displayed to the user through the display interface of the electronic device. Furthermore, the recommended albums can be prioritized for the user based on the shooting time of the images in these recommended albums or the user's interest in the events corresponding to the recommended albums, thereby improving the user's experience in organizing and browsing albums.

[0026] In conjunction with the first aspect, in one possible implementation, the method further includes: in response to a user's operation on a first recommended album in at least one recommended album, the first device displays a second interface, the second interface including multiple sub-albums belonging to the first recommended album, the multiple sub-albums corresponding to multiple subcategories.

[0027] In one implementation, multiple sub-albums of the first recommended album can be obtained by classifying or clustering multiple images in the first recommended album. For example, the images can be classified or clustered based on information of each image in at least one feature dimension in the multiple images in the first recommended album to obtain multiple sub-albums of the first recommended album.

[0028] In another implementation, the multiple images in the first recommended album can be classified or clustered based on information from each image in at least one feature dimension and / or a proximity network to obtain multiple sub-albums of the first recommended album. The proximity network includes information indicating the degree of proximity between multiple objects involved in the multiple images of the first recommended album. In this embodiment, clustering images based on information in the semantic feature dimension can yield more granular sub-albums, further improving the user experience of organizing and browsing albums.

[0029] In conjunction with the first aspect, in one possible implementation, the second interface includes a first control, and the method further includes: in response to a user clicking the first control, saving the first recommended album to the album list of the first device, and adding the images belonging to the first recommended album from the images other than the plurality of images in the gallery to the first recommended album based on the information of each image in the semantic feature dimension.

[0030] Among them, the images belonging to the first recommended album among the images other than the multiple images refer to the images whose semantic feature dimension information is more similar to the images in the first recommended album than a preset similarity.

[0031] In this embodiment, the user can view the recommended albums displayed by the electronic device, and the user can quickly save the recommended albums. When the recommended albums are not saved, the electronic device can cluster only some images in the gallery. After the user triggers the saving of the recommended albums, the electronic device can further add other images in the gallery that belong to the saved recommended albums to the saved recommended albums, which can save the power consumption of the electronic device.

[0032] In conjunction with the first aspect, in one possible implementation, the method further includes: when a first image is added to the first image library, if the similarity between the information of the first image in the semantic feature dimension and the information of the images in the first recommended album in the semantic feature dimension is greater than a preset similarity, then the first image is saved to the first recommended album.

[0033] In this embodiment, after a user saves a recommended album, the electronic device can automatically organize newly added images belonging to that recommended album into the recommended album. Users do not need to manually organize the album continuously, which improves the user's experience in organizing and browsing albums.

[0034] In conjunction with the first aspect, in one possible implementation, the method further includes: generating a second album in response to a user selecting one or more images from the gallery; displaying a third interface in response to a user viewing the second album, the third interface including the one or more images selected by the user, and including one or more candidate images, wherein the one or more candidate images are obtained from images in the gallery that have not been selected by the user, and the similarity between the information of the one or more candidate images in at least one feature dimension and the information of the one or more images selected by the user in at least one feature dimension is greater than a preset similarity; and adding the one or more target images to the second album in response to a user selecting one or more target images from the one or more candidate images.

[0035] In this embodiment of the application, when a user creates a photo album, the user only needs to select a portion of the images belonging to the album from the gallery. The electronic device can move the images not selected by the user into the album based on information of at least one feature dimension of the images selected by the user, and information of at least one feature dimension of the images not selected by the user in the gallery. Furthermore, the user can select images to keep from these automatically moved images, which can improve the efficiency of the user in organizing images.

[0036] In conjunction with the first aspect, in one possible implementation, the method further includes: when a second image is added to the image library, if the similarity between the information of the second image in the semantic feature dimension and the information of the image in the second album in the semantic feature dimension is greater than a preset similarity, then the second image is added to the second album.

[0037] In this embodiment, after a user creates an album, the electronic device can automatically organize newly added images belonging to that album from the gallery into that album. The user does not need to manually organize the album continuously, which can improve the user's experience of organizing and browsing the album.

[0038] In conjunction with the first aspect, in one possible implementation, the at least one feature dimension further includes one or more of a visual feature dimension, a temporal feature dimension, and a spatial feature dimension.

[0039] In conjunction with the first aspect, in one possible implementation, the gallery includes a third image, and the method further includes: determining whether the third image is invalid based on user behavior related to the content of the third image and / or an expiration time determined based on the content of the third image; and when the third image is determined to be invalid, moving the third image to the clean album.

[0040] In this embodiment, the electronic device can automatically clean up invalid images in the image library without requiring manual cleaning by the user, thus providing a better image organization experience.

[0041] Secondly, a method for organizing an image library is provided. The method is applied to a first device and includes: acquiring information of each image in a first album in at least one feature dimension, the at least one feature dimension including a semantic feature dimension, the information of each image in the semantic feature dimension including information obtained by semantic analysis of the content of each image; and generating a title for the first album based on the information of each image in the at least one feature dimension.

[0042] In this embodiment, recommended album titles can be generated based on information in the semantic feature dimension of the image, and personalized and interesting album titles can be generated for the album.

[0043] In conjunction with the second aspect, in one possible implementation, generating a title for the first album based on information of each image in at least one feature dimension includes: generating a title for the first album based on information of each image in at least one feature dimension and an intimacy network, the intimacy network including information for indicating the degree of intimacy between a first object and one or more other objects and / or information for the degree of intimacy between a second object and one or more other objects, wherein the first object, the second object, and the other objects are objects appearing in at least one image in the first album.

[0044] In this embodiment, recommended album titles can be generated based on information in the semantic feature dimension of the image and the proximity network, and personalized and interesting album titles can be generated for the album.

[0045] In conjunction with the second aspect, in one possible implementation, for any one of the plurality of images, the information of the image in the semantic feature dimension includes one or more of the following: a description of the foreground of the image, a description of the background of the image, a description of the relationship between objects in the image, a description of objects in the image, and an overview of the content of the image.

[0046] Thirdly, a method for organizing a photo library is provided, the method being applied to a first device, the photo library of the first device including a third image, the method comprising: determining whether the third image is invalid based on user behavior related to the content of the third image and / or an expiration time determined based on the content of the third image; and when the third image is determined to be invalid, moving the third image to the clean album.

[0047] In this embodiment, the electronic device can automatically clean up invalid images in the image library without requiring manual cleaning by the user, thus providing a better image organization experience.

[0048] Fourthly, a method for organizing an image library is provided, applied to a first device. The method includes: generating a second album in response to a user selecting one or more images from the image library of the first device; displaying a third interface in response to a user viewing the second album, the third interface including the one or more images selected by the user, and including one or more candidate images, wherein the one or more candidate images are obtained from images in the image library that have not been selected by the user, and the similarity between the information of the one or more candidate images in at least one feature dimension and the information of the one or more images selected by the user in the at least one feature dimension is greater than a preset similarity, the at least one feature dimension including a semantic feature dimension, and the information of the image in the semantic feature dimension including information obtained by semantic analysis of the content of the image; and adding the one or more target images to the second album in response to a user selecting one or more target images from the one or more candidate images.

[0049] In this embodiment of the application, when a user creates a photo album, the user only needs to select a portion of the images belonging to the album from the gallery. The electronic device can move the images not selected by the user into the album based on information of at least one feature dimension of the images selected by the user, and information of at least one feature dimension of the images not selected by the user in the gallery. Furthermore, the user can select images to keep from these automatically moved images, which can improve the efficiency of the user in organizing images.

[0050] In conjunction with the fourth aspect, in one possible implementation, the method further includes: when a second image is added to the gallery, if the similarity between the information of the second image in the semantic feature dimension and the information of the image in the second album in the semantic feature dimension is greater than a preset similarity, then the second image is added to the second album.

[0051] In this embodiment, after a user creates an album, the electronic device can automatically organize newly added images belonging to that album from the gallery into that album. The user does not need to manually organize the album continuously, which can improve the user's experience of organizing and browsing the album.

[0052] In conjunction with the fourth aspect, in one possible implementation, the gallery includes a third image, and the method further includes: determining whether the third image is invalid based on the user's behavior related to the content of the third image and / or an expiration time determined based on the content of the third image; and when the third image is determined to be invalid, moving the third image to the clean album.

[0053] In this embodiment, the electronic device can automatically clean up invalid images in the image library without requiring manual cleaning by the user, thus providing a better image organization experience.

[0054] Fifthly, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer program code, and the processor is used to execute the computer program code stored in the memory to implement the method in the first aspect or any possible implementation of the first aspect, or to implement the method in the second aspect or any possible implementation of the second aspect, or to implement the method in the third aspect or any possible implementation of the third aspect, or to implement the method in the fourth aspect or any possible implementation of the fourth aspect.

[0055] In a sixth aspect, a computer-readable storage medium is provided, which stores a computer program or instructions that, when executed, implement the method of the first aspect or any possible implementation thereof, or implement the method of the second aspect or any possible implementation thereof, or implement the method of the third aspect or any possible implementation thereof, or implement the method of the fourth aspect or any possible implementation thereof.

[0056] In a seventh aspect, a chip is provided, wherein instructions are stored therein, which, when executed on a device, cause the chip to perform the method of the first aspect or any possible implementation thereof, or to perform the method of the second aspect or any possible implementation thereof, or to perform the method of the third aspect or any possible implementation thereof, or to perform the method of the fourth aspect or any possible implementation thereof.

[0057] Eighthly, a computer program product is provided, which stores a computer program or instructions that, when executed, implement the method of the first aspect or any possible implementation of the first aspect, or implement the method of the second aspect or any possible implementation of the second aspect, or implement the method of the third aspect or any possible implementation of the third aspect, or implement the method of the fourth aspect or any possible implementation of the fourth aspect. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0059] Figure 2 This is a software structure block diagram of the electronic device provided in the embodiments of this application;

[0060] Figure 3 This is an operational diagram corresponding to a photo album organization method provided in an embodiment of this application;

[0061] Figure 4 This is an operational diagram corresponding to another method for organizing photo albums provided in the embodiments of this application;

[0062] Figure 5 This is an operational diagram corresponding to another method for organizing photo albums provided in the embodiments of this application;

[0063] Figure 6 This is an operational diagram corresponding to another method for organizing photo albums provided in the embodiments of this application;

[0064] Figure 7 This is an operational diagram corresponding to another method for organizing photo albums provided in the embodiments of this application;

[0065] Figure 8 This is a schematic flowchart illustrating a method for organizing images provided in an embodiment of this application;

[0066] Figure 9 This is a schematic flowchart illustrating yet another image organization method provided in the embodiments of this application;

[0067] Figure 10 This is a schematic flowchart illustrating yet another image organization method provided in the embodiments of this application;

[0068] Figure 11 This is a schematic flowchart illustrating another method for image organization provided in the embodiments of this application. Detailed Implementation

[0069] The technical solutions of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0070] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "plural" or "multiple" refers to two or more than two.

[0071] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0072] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0073] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "one embodiment," "some embodiments," "another embodiment," "other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0074] The methods provided in this application can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device.

[0075] For example, Figure 1A schematic diagram of the structure of electronic device 100 is shown. Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0076] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0077] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0078] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0079] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0080] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0081] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge electronic device 100, and can also be used for data transfer between electronic device 100 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0082] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0083] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0084] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0085] The wireless communication function of electronic device 100 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0086] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0087] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0088] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0089] The wireless communication module 160 can provide solutions for wireless communication applications on the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0090] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0091] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0092] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0093] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0094] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0095] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0096] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

[0097] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0098] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0099] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0100] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0101] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0102] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0103] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

[0104] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a telephone call or voice message, the receiver 170B can be brought close to the ear to listen to the voice.

[0105] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 100 may also have three, four, or more microphones 170C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0106] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0107] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. Electronic device 100 can receive button input and generate key signal inputs related to user settings and function control of electronic device 100.

[0108] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0109] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0110] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to make contact with and separate from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 100 uses an embedded SIM (eSIM) card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0111] It should be understood that the phone cards in the embodiments of this application include, but are not limited to, SIM cards, eSIM cards, universal subscriber identity modules (USIM), universal integrated circuit cards (UICC), etc.

[0112] The software system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0113] Figure 2This is a software structure block diagram of an electronic device 100 according to an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer. The application layer may include a series of application packages.

[0114] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0115] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0116] like Figure 2 As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0117] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0118] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0119] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0120] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0121] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0122] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0123] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0124] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0125] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0126] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0127] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0128] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0129] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0130] A 2D graphics engine is a graphics engine for 2D drawing.

[0131] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0132] It should be understood that the technical solutions in the embodiments of this application can be used in systems such as Android, iOS, and HarmonyOS.

[0133] The technical solutions of this application embodiment can be applied to electronic devices with display functions, for example. Exemplarily, they can be applied to televisions, desktop computers, laptops, in-vehicle screens, and portable electronic devices such as mobile phones, foldable screens, tablets, cameras, camcorders, video recorders, watches, and wristbands. They can also be applied to other electronic devices with display functions, and to electronic devices in 5G networks or future evolved public land mobile networks (PLMNs). The main application scenarios can be desktop settings and modification scenarios for electronic devices, such as adjusting icon rounded corners.

[0134] As users take photos with their electronic devices, the number of pictures stored on the devices increases, making it difficult for them to quickly find the pictures they want to view. Therefore, image categorization is particularly important.

[0135] Currently, electronic devices can categorize images based on their source. For example, downloaded images can be grouped into a download album, photos taken with the device's camera into a camera album, and images received through chat software into a received album. However, the number of images from the same source is often large, and these images belong to various categories. Categorizing images by source is far from meeting users' requirements for image organization.

[0136] In some embodiments, electronic devices can classify images into categories such as food, animals, and landscapes. However, in this method, the album classification is generated by a classification algorithm, and the scope of the classification is a closed domain. Different users are interested in different categories. Since users cannot define their own categories, this method cannot meet users' personalized classification needs. Furthermore, this method cannot support finer-grained classification, such as food not distinguishing between hot pot and milk tea, or cats not distinguishing between my cat and a regular cat.

[0137] In some embodiments, users can manually categorize images. For example, a user can create a new album and move images of the same category to that new album. However, album organization is an ongoing process. After storing a new image, the user needs to manually place the new image into the corresponding category album. Users typically organize their albums only once in a while, making it difficult to consistently organize them. Therefore, this method is not very effective for organizing albums and makes it difficult for users to quickly view target images.

[0138] In some other embodiments, users can search for a target image from multiple images using a search function. This requires the user to be very clear about the target image they want to retrieve, and the target image must be describable in words. In this method, the searched target image cannot be persistently saved or updated, and the user needs to repeat the search operation every time they view it, which is cumbersome. Furthermore, this method is not suitable for some images that cannot be described in words.

[0139] Therefore, the current method of categorizing albums cannot meet users' demands for personalized and granular image categories, and it is difficult to enable users to quickly view target images.

[0140] In view of this, embodiments of this application provide a method and electronic device for image organization. Through this method and electronic device, the electronic device can automatically organize images in the gallery according to user preferences and recommend personalized category albums to the user. The user can create a category album with one click based on the personalized category album. After the category album is created, the electronic device can automatically organize the newly saved images corresponding to the category into the category album. This can meet the user's personalized and granular demands for image categories, enable the user to quickly view target images, and truly give the user a low-cost and continuous zero-cost experience.

[0141] Furthermore, through this method and electronic device, after a user creates a category album, the electronic device can automatically organize the newly saved images corresponding to that category into that category album. Users do not need to continuously and manually organize the album, enabling them to quickly view the target images.

[0142] In addition, when users create categorized albums after selecting multiple images, the electronic device can recommend corresponding album names based on the selected images and can also recommend other images to be added, so that users can complete the organization of categorized albums more quickly.

[0143] Using this method and electronic devices, electronic devices can also automatically identify invalid images in the image library, such as screenshots of movie tickets that have already been viewed, images of expired QR codes, etc., and automatically move these invalid images into the junk photo album, thus achieving automatic image cleanup and saving storage space on electronic devices.

[0144] Furthermore, electronic devices can optimize multiple similar photos in the photo library based on the user's historical organization habits. For example, electronic devices can optimize multiple similar group photos based on the close relationship between other users and the device owner.

[0145] For example, combined Figure 3The following is a schematic diagram illustrating the operation of a photo album organization method provided in the embodiments of this application.

[0146] like Figure 3 As shown in (a), when a user opens the gallery application of the electronic device 300, multiple albums are displayed on the display interface of the electronic device 300. Among these multiple albums, one or more recommended albums are obtained by the electronic device 300 through category analysis of multiple images in the gallery. The one or more recommended albums correspond to one or more categories. The one or more recommended albums can be obtained by the electronic device 300 by clustering the images in the gallery based on information of each image in at least one feature dimension. The explanation of the information of each image in at least one feature dimension will be described in detail in subsequent embodiments. The one or more categories can be, for example, categories related to user preferences.

[0147] User preferences may include, for example, user interests and hobbies, documents that the user follows, such as courseware or health tracking, personal items that the user has close relationships with, such as pets, dolls, blankets, figurines, clothing, etc., and may also include other personal items or other content that the user follows. This application does not limit these.

[0148] In some embodiments, the multiple albums displayed on the display interface of the electronic device 300 may also include other albums, such as camera albums, favorite albums, download albums, etc.

[0149] In such Figure 3 In the example shown in (a), the one or more recommended albums may include, for example, an album named "Furry Cutie" and an album named "Health Tracking," wherein the album named "Furry Cutie" includes, for example, one or more photos of the owner's pet cat taken from the gallery, and the album named "Health Tracking" includes, for example, one or more health-related photos taken from the gallery.

[0150] Users can click on any of the recommended albums in the one or more recommended albums to switch the display interface of the electronic device 300 to the viewing interface of that recommended album. This viewing interface can, for example, display one or more sub-albums included in the recommended album, which correspond to one or more subcategories; or, for example, the viewing interface can display one or more photos included in the recommended album.

[0151] For example, users can in such Figure 3 In the interface shown in (a), clicking on the album named "Fluffy Little Cutie" switches the display interface of the electronic device to the one shown in (a). Figure 3The interface shown in (b) is for viewing the album named "Fluffy Cutie". This interface can display one or more sub-albums included in the album named "Fluffy Cutie". These sub-albums may include, for example, an album named "Grown Up" to store photos of "Fluffy Cutie" as an adult; an album named "Childhood" to store photos of "Fluffy Cutie" as a child; an album named "Sleeping" to store photos of "Fluffy Cutie" sleeping; an album named "Eating" to store photos of "Fluffy Cutie" eating; an album named "Playing" to store photos of "Fluffy Cutie" playing; and an album named "Little Cutie and Me" to store photos of "Fluffy Cutie" with the owner.

[0152] In some embodiments, the electronic device 300 can generate an album title based on information from each image in the gallery in at least one feature dimension. In some embodiments, the title of a sub-album can be a humorous title, for example, the title of an album named "Eating" can be replaced with "Just Love Eating"; the title of an album named "Sleeping" can be replaced with "Going to Bed".

[0153] For example, users can in such Figure 3 In the interface shown in (a), by clicking on the album named "Health Tracking", the display interface of the electronic device 300 is switched to the interface for viewing the album named "Health Tracking". In this interface, one or more sub-albums included in the album named "Health Tracking" can be displayed. For example, the album named "Zhang San's Health Tracking" can be used to store Zhang San's health record photos; for example, it can also include the album named "Li Si's Health Tracking" to store Li Si's health record photos.

[0154] In such Figure 3 In the interface shown in (b), the user can, for example, further open any sub-album to view one or more photos included in that sub-album.

[0155] In one example, a user could further click on an album named "Childhood" to switch the electronic device's display to show photos of "Fluffy Little Cutie" as a child.

[0156] In such Figure 3The interface shown in (b) may include a guide box 301, which guides the user to save the current recommended album. The guide box 301 includes a "Magic Organize" control. For example, the guide box may also include the prompt text "Based on the characteristics of the cat photos stored in your gallery, you have been divided into the above categories of albums. After clicking the 'Magic Organize' button, all photos of this cat in the album will be organized into these categories. Photos of this cat that you take or import later will also be automatically organized into the corresponding albums."

[0157] When the user clicks the "Magic Organizer" control, the electronic device 300, based on information from at least one feature dimension of the images in the gallery, organizes all photos of the cat in the gallery into the corresponding sub-album within the album named "Fluffy Cutie," for example, as follows: Figure 3 (b) and as in Figure 3 As shown in (c), when the user clicks the "Magic Organizer" control, the number of photos in the album named "Grown Up" increases from 8 to 94, the number of photos in the album named "Childhood" increases from 8 to 42, the number of photos in the album named "Sleeping" increases from 7 to 17, the number of photos in the album named "Eating" increases from 6 to 23, the number of photos in the album named "Playing" increases from 6 to 39, and the number of photos in the album named "My Little Cutie and Me" increases from 5 to 82.

[0158] For example, users can in such Figure 3 Clicking the album named "Eating" in the interface shown in (c) will switch the display interface of the electronic device 300 to the following: Figure 3 As shown in (d), this interface allows you to view one or more photos in an album named "Eating".

[0159] like Figure 3 As shown in (e), when the user clicks the "Magic Organizer" control, the electronic device 300 saves the album named "Fluffy Little Cutie" from the recommended album list to the My Album list. In addition, new recommended albums can also appear in the recommended album list, such as the album named "Poetry and Distant Places", which is used to organize travel photos from the gallery into this album.

[0160] Information about an image in at least one feature dimension may include information about the image in the semantic feature dimension. For example, information about the image in the semantic feature dimension may include information obtained by semantic analysis of the overall graphics and / or graphics segmentation results in the image, or information obtained by semantic analysis of the text in the image.

[0161] In this embodiment, based on the information obtained from semantic analysis of each image in the image library, an album named "Fluffy Little Cutie" can be clustered. The information obtained from semantic analysis of each image may include, for example, a description of the foreground of the image, a description of the background of the image, a description of the objects in the image, a description of the relationships between the objects in the image, and a summary of the content of the image. In this embodiment, the information obtained from semantic analysis of images belonging to the album named "Fluffy Little Cutie" may include, for example, information such as: the foreground of the image includes a cat, the physical characteristics of the cat in the image, the behavior of the cat in the image, and the relationship between the cat in the image and other objects in the image. Through this information, one can know whether this is my cat, whether it is an adult cat or a kitten, what the cat is doing in the image (e.g., eating, sleeping, playing, etc.), and the relationship information between the cat and other objects in the image (e.g., the owner).

[0162] In this embodiment, the album title can be generated based on the information of each image in the gallery in the semantic feature dimension. The description of the information of the images in the semantic feature dimension will be described in detail in subsequent embodiments.

[0163] For example, combined Figure 4 The following is a schematic diagram illustrating another method for organizing a photo album provided in the embodiments of this application.

[0164] like Figure 4 As shown in (a), when a user opens the gallery application of the electronic device 400, multiple albums are displayed on the display interface of the electronic device 400. Among these multiple albums, there are one or more recommended albums. These one or more recommended albums can be obtained by the electronic device 400 clustering the images in the gallery based on information of each image in at least one feature dimension. The explanation of the information of each image in at least one feature dimension will be described in detail in subsequent embodiments. Based on the gallery, the one or more recommended albums correspond to one or more categories.

[0165] In some embodiments, the multiple albums displayed on the display interface of the electronic device 400 may also include other albums, such as camera albums, download albums, etc.

[0166] In such Figure 4In the example shown in (a), the one or more recommended albums may include, for example, an album named "Zhang San's Health Book", an album named "Li Si's Health Book", an album named "My Cat", and an album named "Other Cats". The album named "Zhang San's Health Book" may include, for example, one or more photos related to Zhang San's health, clustered from photos in the gallery; the album named "Li Si's Health Book" may include, for example, one or more photos related to Li Si's health, clustered from photos in the gallery; the album named "My Cat" may include, for example, one or more photos of the owner's cat, clustered from photos in the gallery; and the album named "Other Cats" may include, for example, one or more photos of cats other than the owner's cat, clustered from photos in the gallery.

[0167] In such Figure 4 In the example shown in (a), the recommended albums are arranged in the following order: album named "Zhang San's Health Book", album named "Li Si's Health Book", album named "My Cat", and album named "Other Cats".

[0168] In some embodiments, recommended albums containing newly captured images by electronic devices are given priority.

[0169] In some embodiments, the order of recommended albums can be determined based on the user's preference for the events corresponding to these recommended albums, wherein the user's preference for the events corresponding to these recommended albums can be determined by analyzing the clustering results of the images.

[0170] Based on the above recommended album arrangement order, such as Figure 4 (b) and as in Figure 4 As shown in (c), after taking a photo A of Li Si's medical report using the electronic device, the display interface of the electronic device is returned to the display interface of multiple albums. Since the album named "Li Si's Health Book" has added a new photo A of Li Si's medical report, the recommended album containing the newly taken image by the electronic device becomes the album named "Li Si's Health Book". The order in which the multiple recommended albums are recommended changes, and the album named "Li Si's Health Book" is recommended first. That is, the order in which the multiple recommended albums are recommended is as follows: album named "Li Si's Health Book", album named "Zhang San's Health Book", album named "My Cat", and album named "Other Cats".

[0171] For example, users can in such Figure 4 In the interface shown in (c), clicking on the album named "Li Si Health Book" switches the display interface of the electronic device to the one shown in the image. Figure 4The interface shown in (d) displays the photo album named "Li Si's Health Book". This interface displays one or more photos related to Li Si's health, and shows that the album includes a newly taken photo A of Li Si's medical report. In other words, newly added images to the gallery can be automatically added to their respective albums.

[0172] In such Figure 4 The interface shown in (d) may include a guide box 401, which guides the user to save the current recommended album. The guide box 401 includes a "Magic Organizer" control. The guide box may also include prompt text such as "After clicking the 'Magic Organizer' button, an album will be generated. Photos you take or import that contain the album's features will be automatically organized into the album."

[0173] When the user clicks the "Magic Organizer" control, the electronic device 400 organizes all the photos related to Li Si's health in the gallery into an album named "Li Si's Health Book".

[0174] like Figure 4 As shown in (e), when the user clicks the "Magic Organizer" control, the electronic device 400 saves the album named "Li Si Health Book" from the recommended album list to the My Album list. In addition, the recommended album list can also display new recommended albums, such as the album named "Poetry and Distant Places", which is used to organize travel photos from the gallery into this album.

[0175] In this embodiment, an album named "Li Si's Health Book" can be clustered based on the information obtained from semantic analysis of the text in each image in the gallery. The information obtained from semantic analysis of the text in each image may include, for example, that the image is related to a health report, that the health report belongs to Li Si, the date of the health report, and the examination site corresponding to the health report.

[0176] In this embodiment of the application, even if the newly taken photo is obscured, the electronic device can store the newly taken photo in the corresponding album based on information in at least one feature dimension of the photo (e.g., content relevance, time similarity, location consistency, etc.).

[0177] For example, combined Figure 5 The following is a schematic diagram illustrating another method for organizing a photo album provided in the embodiments of this application.

[0178] like Figure 5As shown in (a), when a user opens the gallery application of electronic device 500, multiple "My Albums" are displayed on the display interface of electronic device 500, such as camera album, download album, etc.

[0179] Responding to user actions such as Figure 5 Clicking the control 501 used to create a photo album in the interface shown in (a) will switch the display interface of the electronic device 500 to the following: Figure 5 The interface shown in (b) displays an album creation box 502, which is used by the user to create an album. For example, the user can use the album creation box 502 to determine the name of the album to be created, and the user can use the album creation box 502 to select the storage location of the album to be created. The album creation box 502 may also include a "cancel" control and an "OK" control, which are used by the user to cancel the creation of the album or confirm the creation of the album during the album creation process.

[0180] For example, such as Figure 5 As shown in (b), the user enters the new album name "Advanced Mathematics" in the album creation box 502, selects the storage location of the new album as the device, and clicks the "OK" control. At this time, the display interface of the electronic device switches to the one shown in (b). Figure 5 The photo selection interface shown in (c) allows users to select one or more pictures to be placed in a newly created album named "Advanced Mathematics".

[0181] When users are like Figure 5 In the interface shown in (c), after selecting "Advanced Mathematics Photo 1", "Advanced Mathematics Photo 2", and "Advanced Mathematics Photo 3" and clicking the "Finish" control, the display interface of the electronic device 500 can be switched to, for example, as shown in (c). Figure 5 The interface shown in (d) displays multiple My Albums, including a newly created album named "Advanced Mathematics".

[0182] For example, when a user is in such Figure 5 Clicking on the album named "Advanced Mathematics" in the interface shown in (d) will switch the display interface of the electronic device 500 to the following: Figure 5 The photo viewing interface shown in (e) displays "Advanced Mathematics Photo 1", "Advanced Mathematics Photo 2", and "Advanced Mathematics Photo 3".

[0183] In some embodiments, newly captured images by an electronic device are automatically organized into the album associated with the image.

[0184] For example, such as Figure 5 (f) in the middle, such as Figure 5 (g) and such Figure 5As shown in (h), after a user creates an album named "Advanced Mathematics", when the user takes a new photo named "Advanced Mathematics Photo 4" using the electronic device 500, the photo is automatically added to the album named "Advanced Mathematics". The user can then view the newly taken photo "Advanced Mathematics Photo 4" by opening the album named "Advanced Mathematics".

[0185] For example, taking a photograph as an example, combined with Figure 6 The following is a schematic diagram illustrating another method for organizing a photo album provided in the embodiments of this application.

[0186] like Figure 6 As shown in (a), the display interface of the electronic device 600 displays an interface for editing all photos in the gallery. Through this interface, the user can view all the photos included in the gallery, and each photo is associated with controls 601 and 602. Control 601 is used to zoom in on the photo, and control 602 is used to select the photo.

[0187] For example, when a user is in such Figure 6 When photos “Advanced Mathematics Photo 1”, “Advanced Mathematics Photo 2”, and “Advanced Mathematics Photo 3” are selected in (a) and the “New” control is clicked, the display interface of the electronic device 600 switches to the following format: Figure 6 The interface shown in (b) displays an album creation box 603, which is used by the user to create an album based on selected photos. For example, the user can use the album creation box 603 to determine the name of the album to be created, and the user can use the album creation box 603 to select the storage location of the album to be created. The album creation box 603 may also include a "cancel" control and an "OK" control, which are used by the user to cancel the creation of the album or confirm the creation of the album during the album creation process.

[0188] For example, such as Figure 6 As shown in (b), the user enters the new album name "Advanced Mathematics" in the album creation box 603, selects the device as the storage location for the new album, and clicks the "OK" control. At this time, the display interface of the electronic device switches to the one shown in (b). Figure 6 The album viewing interface shown in (c) shows that the photos selected by the user, "Advanced Mathematics Photo 1", "Advanced Mathematics Photo 2", and "Advanced Mathematics Photo 3", are located in the newly created album named "Advanced Mathematics".

[0189] In addition, the interface may include one or more candidate photos, which may be obtained by the electronic device 600 from the unselected photos in the gallery based on information of the photo selected by the user in at least one feature dimension and information of the unselected photos in the gallery in at least one feature dimension.

[0190] For example, each of the one or more candidate photos is associated with a control 602 for selecting the photo. The user can select the photo from the one or more candidate photos on the interface and decide to add it to the album named "High Mathematics". Then, the user clicks the "Done" control and the selected candidate photo will be added to the album named "High Mathematics".

[0191] In some embodiments, the electronic device can automatically identify expired photos in the gallery based on the user's photo-related behavior and / or the expired time of the photos determined based on the content of the photos, and automatically organize the identified expired photos into a junk album, through which the user can clean up expired photos in a unified manner.

[0192] Among them, invalid photos can include invalid QR code photos, expired movie ticket photos, navigation map photos of the destination already reached, expired ticket photos, outdated game guide photos, etc.

[0193] For example, combined Figure 7 The following is a schematic diagram illustrating another method for organizing a photo album provided in the embodiments of this application.

[0194] like Figure 7 As shown in (a), when a user opens the gallery application of electronic device 700, multiple "My Albums" are displayed on the display interface of electronic device 700, such as camera album, downloaded album, junk album, etc.

[0195] For example, such as Figure 7 (a) and Figure 7 As shown in (b), with the current date being October 15, 2024, the album named "Camera" contains 33 photos, and the album named "Trash Album" contains 0 photos. When a user views the album named "Camera" using electronic device 700, multiple photos taken by electronic device 700 are displayed on the display interface of electronic device 700. These multiple photos include "Movie Ticket Photo for the 21st" and "Ticket Photo for the 20th". The "Movie Ticket Photo for the 21st" refers to a photo of a movie ticket with a viewing date of the 21st, and the "Ticket Photo for the 20th" refers to a photo of a ticket with an available date of the 20th.

[0196] Based on this, for example, such as Figure 7 (c) and Figure 7As shown in (d), with the current date being October 21, 2024, the album named "Camera" contains 32 photos, the album named "Trash Album" contains 1 photo, and the "Ticket Photo No. 20" originally located in the album named "Camera" has been automatically moved to the album named "Trash Album" by the electronic device.

[0197] Based on this, for example, such as Figure 7 (e) and Figure 7 As shown in (f), with the current date being October 22, 2024, the album named "Camera" contains 31 photos, the album named "Trash Album" contains 2 photos, and the "Movie Ticket Photo of the 21st" originally located in the album named "Camera" has been automatically moved to the album named "Trash Album" by the electronic device.

[0198] Similarly, electronic devices can automatically organize similar photos in the gallery. For example, electronic devices can automatically organize similar photos in the gallery into an album. Electronic devices can also recommend preferred photos based on the owner's habits, the closeness of the people, etc., or electronic devices can automatically keep preferred photos and automatically move non-preferred photos to the junk album.

[0199] In some embodiments, for screenshots or downloaded images, the electronic device can first classify and identify the information of the image, and different classifications correspond to different cleaning strategies.

[0200] In one example, for map navigation images, the electronic device can compare the start and end point information on the image with the user's current geographical location information. If the user has already passed through the start and end points, the image is added to the junk photo album.

[0201] In one example, for images related to ticket scanning, the electronic device can identify the latest time on the image. If the current time is already later than the latest time, it determines that the timeout has occurred and adds the image to the junk photo album.

[0202] In one example, for shared images, if the electronic device detects that the image has been selected in other applications, it adds it to the junk album.

[0203] In some examples, for other categories of images, electronic devices can analyze the validity period of images using a large model, and if the validity period obtained based on the large model has expired, the image is added to the junk album.

[0204] In some embodiments, for captured images, the electronic device can identify duplicate and similar images.

[0205] In one example, for a duplicate image, the electronic device can directly add the duplicate image to the junk photo album.

[0206] In one example, for similar photos, the electronic device can retain similar images that are highly intimate with the user and have high portrait aesthetic scores and clarity, based on the user's closeness to other people or things, while adding other similar images to the junk photo album.

[0207] In some embodiments, the electronic device may increase the category weight of images actively deleted by the user when cleaning up images; similarly, the electronic device may decrease the category weight of images removed from the junk album by the user when cleaning up images.

[0208] In some embodiments, the expiration date of an image can be inferred by combining information such as the user's behavioral perception data, the time and location of the image being taken, the source of the image, and the content of the image layout. Based on the expiration date, the user can be promptly reminded to clean up the photo album.

[0209] For example, Figure 8 A schematic flowchart of an image organization method 800 provided in an embodiment of this application is shown. Figure 8 As shown, the method 800 includes:

[0210] S801: Obtain information about each image in at least one feature dimension from a plurality of images in the image library, wherein the information about each image in at least one feature dimension includes information about each image in the semantic feature dimension.

[0211] The information in the semantic feature dimension of each image includes information obtained by semantic analysis of the content of each image. For example, it may include information obtained by semantic analysis of the overall graphics and / or graphics segmentation results in each image, as well as information obtained by semantic analysis of the text in each image.

[0212] In some embodiments, for any one of a plurality of images, the information of the image in the semantic feature dimension may include, for example, one or more of the following: a description of the foreground of the image, a description of the background of the image, a description of the relationship between objects in the image, a description of objects in the image, and an overview of the image content.

[0213] The description of the background of an image may include descriptions of people, objects, or the environment in the background, as well as descriptions of the relationships between them. For example, the background of an image may contain several children playing on the beach, a sunny and calm day, or a stormy and rainy day.

[0214] The description of the relationship between objects in an image is as follows: if the image includes a person and a cat, the relationship between the objects in the image could be described as a person holding a cat; if the image includes an adult and a child, the relationship between the objects in the image could be described as an adult holding a child. The objects mentioned here can be objects in the foreground or objects in the background.

[0215] The description of the object in the image may include one or more of the following: description of the object's behavior, description of the object's appearance, description of the object's expression, description of the object's clothing. In addition, it may also be other descriptions of the object, which are not limited in this application.

[0216] For example, the summary of the image content could be described as follows: if the image shows primary school students having a sports meet, the summary could be "school sports meet"; if the image shows a couple at the beach with the girl wearing a long dress, the summary could be "playing at the beach"; if the image shows several teachers leading a group of students in school uniforms walking enthusiastically in a park with everything coming back to life and the sun shining brightly in the background, the summary could be "school spring outing"; and if the image shows elderly people, young people, and children happily eating hot pot together, the summary could be "family gathering".

[0217] Understandably, with Figure 4 Taking the illustrated embodiment as an example, an album named "Li Si's Health Book" can be clustered based on the information obtained from semantic analysis of the text in each image in the gallery. The information obtained from semantic analysis of the text in each image may include, for example, that the image is related to a health report, that the health report belongs to Li Si, the date of the health report, and the corresponding examination site of the health report.

[0218] Understandably, with Figure 3 Taking the illustrated embodiment as an example, based on the information obtained from semantic analysis of the content of each image in the image library, an album named "Fluffy Little Cutie" can be clustered. In this embodiment, the information obtained from semantic analysis of the content of the images belonging to the album named "Fluffy Little Cutie" may include, for example, information such as: the foreground of the image includes a cat, the physical characteristics of the cat in the image, the behavior of the cat in the image, and the relationship between the cat in the image and other objects in the image. Through this information, one can know whether this is my cat, whether it is an adult cat or a kitten, what the cat in the image is doing (e.g., eating, sleeping, playing, etc.), and the relationship information between the cat and other objects in the image (e.g., the owner).

[0219] In some embodiments, the information of each image in at least one feature dimension may further include information of each image in the visual feature dimension and / or in the temporal feature dimension and / or in the spatial feature dimension.

[0220] In some embodiments, multiple images in the gallery are first clustered based on the semantic feature dimension information of each image in the gallery to generate one or more recommended albums; then, multiple images in the gallery are clustered a second time based on the visual feature dimension information and / or the temporal feature dimension information and / or the spatial feature dimension information of each image in the gallery to update one or more recommended albums.

[0221] In some embodiments, multiple images in the gallery are clustered based on information about each image in the gallery in the semantic feature dimension, as well as information in the visual feature dimension and / or information in the temporal feature dimension and / or information in the spatial feature dimension, to generate one or more recommended albums.

[0222] In some embodiments, the image captioning capability of the edge-side image-text large model can be used to obtain one or more of the information of the image in the semantic feature dimension, the visual feature dimension, the temporal feature dimension, and the spatial feature dimension. The input of the edge-side image-text large model is an image, and the output is one or more of the information of the image in the semantic feature dimension, the visual feature dimension, the temporal feature dimension, and the spatial feature dimension.

[0223] In one implementation, information about the semantic features of an image is obtained by performing object segmentation, content description, and combining the image with its corresponding classification labels.

[0224] In one implementation, information about the image's visual features is obtained by parsing the image's metadata.

[0225] Information about an image in the visual feature dimension can include, for example, facial information, the color, shape, and movement of the subject in the image.

[0226] In one implementation, information about the image in the temporal feature dimension and / or spatial feature dimension is obtained by extracting the image's shooting time, shooting location, and / or image source.

[0227] The information in the temporal dimension of the image can include, for example, the time when the image was captured, and the information in the spatial dimension of the image can include, for example, the location where the image was captured, and the source information of the image.

[0228] S802: Cluster multiple images in the gallery based on information of each image in at least one feature dimension to generate one or more recommended albums.

[0229] In some embodiments, the method 800 may further include: constructing a proximity network corresponding to the plurality of images by analyzing the clustering results of the plurality of images in the gallery, the proximity network being used to indicate the degree of proximity between a first object and one or more other objects and / or the degree of proximity between a second object and one or more other objects, wherein the first object, the second object and the other objects are objects appearing in at least one of the plurality of images.

[0230] In some embodiments, the first object can be any one of a person, an animal, or an item, and the second object can be any one of a person, an animal, or an item.

[0231] The method is applied to a first device, and the first object may be, for example, a first person, which may be, for example, a user of the first device.

[0232] In some embodiments, based on the constructed intimacy network, for one or more recommended albums obtained through clustering, the first device may prioritize displaying the recommended albums with higher intimacy levels to the user.

[0233] In some embodiments, based on the constructed affinity network, images with low affinity can be prioritized for cleaning during image cleanup.

[0234] In some embodiments, the first object is a first person. Based on the constructed intimacy network, the relationships between the first person in the image and other people can be obtained, such as family relationships, friend relationships, colleague relationships, etc.; the relationships between the first person in the image and animals can also be obtained, such as my cat, etc.; the relationships between the first person and objects can also be obtained, such as my car, etc.

[0235] In some embodiments, the clustered albums can be further classified or clustered based on the constructed intimacy network. In one example, 100 images are clustered to obtain 10 recommended albums, and any one of these 10 recommended albums can be further classified or clustered into multiple sub-albums based on the constructed intimacy network.

[0236] In some embodiments, multiple albums obtained from clustering can be merged according to the constructed affinity network. In one example, 100 images are clustered to obtain 10 recommended albums, and two or more of these 10 recommended albums can be merged into one recommended album according to the constructed affinity network.

[0237] In some embodiments, titles for multiple clustered albums can be generated based on the constructed intimacy network. In one example, 100 images are clustered to obtain 10 recommended albums. Any one of these 10 recommended albums can be further classified or clustered into multiple sub-albums based on the constructed intimacy network. Two or more of the 10 recommended albums can be merged into one recommended album based on the constructed intimacy network. Based on the constructed intimacy network, titles for the aforementioned recommended albums, sub-albums, and merged recommended albums can be generated.

[0238] In some embodiments, multiple images are clustered based on information of at least one feature dimension corresponding to each image in the multiple images. The at least one feature dimension includes one or more of semantic feature dimension, visual feature dimension, temporal feature dimension and spatial feature dimension, and then a proximity network is constructed based on the obtained clustering results.

[0239] In some embodiments, the method 800 may further include: generating titles for one or more recommended albums based on information about each image in the gallery in at least one feature dimension.

[0240] For example, the title of the first recommended album can be generated based on the semantic feature dimension information of each image in the first recommended album from one or more recommended albums.

[0241] In one implementation, information about each image in the gallery in at least one feature dimension can be input into a large model, which then outputs the titles of one or more recommended albums.

[0242] It is understandable that after clustering multiple images to obtain multiple categories, taking category 1 as an example, the title of the recommended album corresponding to category 1 is generated based on the information of the image corresponding to category 1 in at least one feature dimension.

[0243] For example, taking recommended album 1 in one or more recommended albums as an example, the title of recommended album 1 can be generated based on information of the images belonging to recommended album 1 in at least one feature dimension.

[0244] For example, with Figure 3 As shown in the example, the album title generated based on information from each image in the gallery in at least one feature dimension may include "Furry Cutie", "Health Tracking", "Grown Up", "Cutie and Me", "Eating", "Sleeping", etc.

[0245] In some embodiments, the title of at least one of the recommended albums can be generated based on the intimacy network.

[0246] In some embodiments, titles for one or more recommended albums can be generated based on information in at least one feature dimension for each image in the gallery and an intimacy network. For example, after clustering multiple images to obtain multiple categories, taking category 1 as an example, the title of the recommended album corresponding to category 1 is generated based on information in at least one feature dimension for the images corresponding to category 1 and information included in the intimacy network to indicate the intimacy between objects in the images corresponding to category 1.

[0247] In some embodiments, after generating one or more recommended albums, the first device may display a first interface, which includes at least one of the one or more recommended albums.

[0248] In some embodiments, at least one recommended album may be multiple recommended albums, and the display order of these multiple recommended albums on the first interface may be determined, for example, based on the image capture time of the recommended album. The image capture time of the recommended album may refer to the capture time of the most recently captured image in the recommended album, or it may be described as the display order of the multiple recommended albums on the first interface being determined, for example, based on the update time of the recommended album.

[0249] For example Figure 4 Taking the illustrated embodiment as an example, in such... Figure 4 In the interface shown in (a), the display order of multiple recommended albums, from first to last, is: album named "Zhang San's Health Book", album named "Li Si's Health Book", album named "My Cat", and album named "Other Cats". After the first device takes a new picture of "Li Si's Medical Report A", the album named "Li Si's Health Book" adds the picture "Li Si's Medical Report A". At this time, since the picture "Li Si's Medical Report A" is the most recently taken picture, the display order of the album named "Li Si's Health Book" changes to the first one displayed (e.g., Figure 4 (as shown in (c)).

[0250] In some embodiments, at least one recommended album may be multiple recommended albums. Furthermore, by analyzing the clustering results of multiple images in the gallery, the user's level of interest in the events corresponding to the generated recommended albums can be obtained, and the display order of the multiple recommended albums on the first interface can be determined based on the user's level of interest in the events corresponding to the generated recommended albums.

[0251] In one possible implementation, if a user has a strong interest in the event corresponding to the generated recommended album (for example, the time span between the latest and earliest shooting events of the image corresponding to the event is greater than 1, and the shooting frequency is higher than 1), the recommended album corresponding to the event can be recommended to the user first when preset conditions are met. The preset conditions can be, for example, when the recommended album is generated, when the user opens the gallery, or a preset time point, such as 6 pm.

[0252] For example Figure 3 Taking the illustrated embodiment as an example, based on the clustering results of the images in the gallery, it is found that the user's love for cats is greater than their concern for health status. When recommending albums to users, the album named "Fluffy Cutie" is displayed before the album named "Health Tracking".

[0253] In some embodiments, the clustering results of multiple images in the gallery can be analyzed to discover user preferences such as anniversaries and places of residence, and the display order of at least one recommended album on the first interface can be determined based on the user's preferences, for example, the recommended album corresponding to the user's preferences can be displayed first.

[0254] Optionally, after clustering to obtain one or more recommended albums, not all of these albums are immediately displayed to the user. The timing of displaying each recommended album can be different; it can be displayed to the user at the recommended timing corresponding to the album. The recommended timing for a particular album can be related to factors such as the current time, the user's location, or currently occurring events.

[0255] In some embodiments, the timing of recommending albums on the first interface can be determined based on the current time. The electronic device can prioritize recommending albums that match the current time to the user. For example, when the first device recognizes that the current time is the Dragon Boat Festival, it will prioritize displaying albums related to the Dragon Boat Festival on the interface.

[0256] In some embodiments, the display order of recommended albums on the first interface can be determined based on the user's current location. The first device can prioritize recommending albums that match the user's current location. For example, when the first device detects that the user has arrived in Huangshan, it will prioritize displaying albums related to Huangshan on the interface.

[0257] In some embodiments, the display order of recommended albums on the first interface can be determined based on the currently occurring event. The electronic device can prioritize recommending albums that match the currently occurring event to the user. For example, when the first device detects that the user is cycling through a sports and health program, or detects that the user has taken a photo of cycling through the first device, albums related to the cycling event are prioritized to be displayed on the interface.

[0258] In some embodiments, recommended albums can also be displayed to the user in the form of cards on the desktop of the first device. For example, images from recommended albums can be displayed on a gallery card displayed on the desktop of the first device. The display method may include, for example, playing in turn, displaying single images, etc.

[0259] In some embodiments, the method 800 may further include: in response to a user's operation on a first recommended album in at least one recommended album, the first device displays a second interface, the second interface including a plurality of sub-albums belonging to the first recommended album, the plurality of sub-albums corresponding to a plurality of subcategories.

[0260] In one implementation, multiple sub-albums of the first recommended album can be obtained by classifying or clustering multiple images in the first recommended album. For example, the images can be classified or clustered based on the information of each image in the semantic feature dimension in the first recommended album to obtain multiple sub-albums of the first recommended album.

[0261] In another implementation, the multiple images in the first recommended album can be classified or clustered based on the information of each image in the semantic feature dimension and / or the intimacy network corresponding to the first recommended album to obtain multiple sub-albums of the first recommended album. The intimacy network corresponding to the first recommended album is used to indicate the degree of intimacy between the multiple objects involved in the multiple images in the first recommended album.

[0262] Among them, the user's operation on the first recommended album can be, for example, the user's touch operation on the first recommended album, such as a click operation.

[0263] For example, with Figure 3 The illustrated embodiment is an example, responding to the user in such... Figure 3 The interface shown in (a) represents a touch operation on an album named "Fluffy Little Cuties". The first device displays the following: Figure 3 The interface shown in (b) includes multiple sub-albums belonging to the album named "Fluffy Little Cutie". These sub-albums may include, for example, an album named "Grown Up", an album named "Childhood", an album named "Sleeping", an album named "Eating", an album named "Playing", and an album named "Little Cutie and Me".

[0264] In some embodiments, the multiple images used for clustering to generate one or more recommended albums may be a subset of images from a gallery, such as a subset of images selected by the user from the gallery. For example, the second interface described above may include a first control, and the method 800 may further include: in response to the user clicking the first control, saving the first recommended album to the album list of the first device.

[0265] Specifically, after the user clicks the first control, the first device can add other images belonging to the first recommended album from the gallery to the first recommended album based on information about each image in at least one feature dimension. For example, an image belonging to the first recommended album can be an image whose information in the semantic feature dimension is more similar to that of an image in the first recommended album than a preset similarity.

[0266] In some embodiments, the method 800 may further include: when a first image is added to the image library, if the similarity between the information of the first image in at least one feature dimension and the information of images in the first recommended album in at least one feature dimension is greater than a preset similarity, then the first image is added to the first recommended album. In one example, the information of the first image in the semantic feature dimension can be compared with the information of each image in the first recommended album in the semantic feature dimension to obtain the similarity between the information of the first image in the semantic feature dimension and the information of each image in the first recommended album in the semantic feature dimension. This allows obtaining multiple similarities corresponding to multiple images in the first recommended album. Then, the mean or median of these similarities is calculated. If the calculated mean or median is greater than the preset similarity, then the first image is added to the first recommended album.

[0267] In one example, the similarity of the semantic feature information of the first image with the semantic feature information of each image in the first recommended album can be compared with a preset similarity. If the similarity is greater than the preset similarity, the image in the first recommended album is marked as class A. The first image and each image in the first recommended album are compared in this way. If the proportion of the number of images marked as class A exceeds the proportion threshold (e.g., 50%, 80%), it can be considered that the semantic feature information of the first image is more similar to the semantic feature information of the images in the first recommended album than the preset similarity, and the first image can be added to the first recommended album.

[0268] In one example, the mean or median of semantic feature information for multiple images in the first recommended album can be obtained first. Then, the semantic feature information of the first image can be compared with the mean or median. If the similarity between the semantic feature information of the first image and the mean or median is greater than a preset similarity, the first image can be added to the first recommended album. Figure 4 Taking the illustrated embodiment as an example, if the album named "Li Si's Health Report" does not contain the image "Li Si's Medical Report A", after the user takes a picture of the image "Li Si's Medical Report A" using the first device, the image "Li Si's Medical Report A" is added to the gallery of the first device. At this time, since the similarity between the information of the image "Li Si's Medical Report A" in the semantic feature dimension and the information of the image in the album named "Li Si's Health Report A" in the semantic feature dimension is greater than the preset similarity, the image "Li Si's Medical Report A" is added to the album named "Li Si's Health Report A".

[0269] It can be understood that S801 and S802 can both be executed on the device side; or they can be partially executed on the device side and partially executed on the cloud side; or they can be entirely executed on the cloud side.

[0270] In this embodiment, images in the image library can be clustered based on information in the semantic feature dimension to generate one or more recommended albums, and at least one of the recommended albums can be recommended to the user. This eliminates the need for the user to manually organize the images and allows the user to quickly find the target image.

[0271] Furthermore, newly added images in the gallery can be automatically clustered into the corresponding albums, eliminating the need for users to manually organize them periodically, thus providing users with a convenient and efficient image browsing experience.

[0272] For example, Figure 9 A schematic flowchart of another image organization method 900 provided in an embodiment of this application is shown. Figure 9 As shown, the method 900 includes:

[0273] S901: In response to a user's action of selecting one or more images from a gallery, a second album is generated, which includes the one or more images selected by the user.

[0274] S902: In response to the user's operation of viewing the second album, the first device displays a third interface, which includes one or more images selected by the user, and one or more images to be selected.

[0275] The one or more candidate images are obtained from images in the image library that have not been selected by the user, and the similarity between the information of the one or more candidate images in at least one feature dimension and the information of the one or more images selected by the user in at least one feature dimension is greater than a preset similarity.

[0276] Among them, the information of the image in at least one feature dimension includes the information of the image in the semantic feature dimension, and the interpretation of the information of the image in the semantic feature dimension is as follows: Figure 8 The embodiments shown are described in detail, and for the sake of brevity, they will not be repeated here.

[0277] In some embodiments, the information of an image in at least one feature dimension may further include information of the image in the visual feature dimension and / or information of the image in the temporal feature dimension and / or information of the image in the spatial feature dimension, wherein the interpretation of the information of the image in the visual feature dimension, the information in the temporal feature dimension, and the information in the spatial feature dimension is... Figure 8 The embodiments shown are described in detail, and for the sake of brevity, they will not be repeated here.

[0278] S903: In response to a user's operation of selecting one or more target images from the one or more candidate images, the first device adds the one or more target images to the second album.

[0279] For the interface diagrams corresponding to the embodiments of this application, please refer to... Figure 6 The example shown.

[0280] In some embodiments, the method 900 may further include: when a second image is added to the gallery, if the similarity between the information of the second image in at least one feature dimension and the information of an image in the second album in at least one feature dimension is greater than a preset similarity, then the second image is added to the second album.

[0281] by Figure 5 Taking the illustrated embodiment as an example, if the album named "High Mathematics" does not contain the image "High Mathematics Photo 4", after the user takes the image "High Mathematics Photo 4" using the first device, the image "High Mathematics Photo 4" is added to the gallery of the first device. At this time, since the similarity between the information of the image "High Mathematics Photo 4" in the semantic feature dimension and the information of the image in the album named "High Mathematics" in the semantic feature dimension is greater than the preset similarity, the image "High Mathematics Photo 4" is added to the album named "High Mathematics".

[0282] In some embodiments, the first device can optimize the filtering criteria for the selected images based on the user's retention of the selected images, so that the selected images subsequently displayed to the user by the first device become increasingly closer to the user's actual needs.

[0283] In this embodiment, for user-created albums, the electronic device can automatically add images whose semantic feature dimension information in the image library is greater than a preset similarity to the images in the user-created album, based on the semantic feature dimension information of the images in the image library. In this way, when creating an album, the user only needs to select some images in the image library to achieve automatic organization of all images in the image library without the need for manual organization by the user, which can bring the user a convenient and efficient image browsing experience.

[0284] For example, Figure 10 A schematic flowchart of another image organization method 1000 provided in an embodiment of this application is shown. Figure 10 As shown, the method 1000 is applied to a first device, the first device's image library including a third image, and the method 1000 includes:

[0285] S1001: Determine whether the third image is invalid based on the user's behavior related to the content of the third image and / or the invalidation time determined based on the content of the third image.

[0286] In some embodiments, user behaviors related to the content of the third image may include, for example, changes in the user's location or ticket purchase behavior. The expiration time of the third image may be determined, for example, based on the storage time limit corresponding to the image type of the third image and / or the validity period identified from the content of the third image. In one example, the third image is an image used to represent a navigation route. The first device may identify the starting point, waypoints, and / or destination in the screenshot. When the first device detects that its current location information has reached the destination of the navigation route, the first device may determine that the third image has expired.

[0287] In one example, the third image is an image related to a movie ticket or admission ticket. The first device can identify the validity period of the movie ticket or admission ticket. When the first device detects that the current time has exceeded the validity period of the movie ticket or admission ticket, the first device can determine that the third image is invalid.

[0288] In one example, the third image is a temporary image, such as a daily work plan picture. The temporary image is set to be saved for 2 days. When the first device detects that the third image has been saved in the first device's image library for more than 2 days, the first device can determine that the third image is invalid.

[0289] In one example, the third image is used to represent the walkthrough for game 1 version 1. The first device can determine that the third image is invalid when it is detected that the user has saved the walkthrough for a version of game 1 that is higher than version 1, or when it is detected that the user has uninstalled game 1 version 1.

[0290] S1002: When it is determined that the third image is invalid, move the third image to the clean album of the first device.

[0291] In some embodiments, similar images in the gallery can also be automatically cleaned up. For example, similar images with low resolution can be cleaned up based on their resolution; or similar images can be cleaned up based on a network of resolution and proximity.

[0292] The proximity network can be, for example, derived from clustering multiple images in a database based on information in at least one feature dimension. This at least one feature dimension includes one or more of semantic, visual, temporal, and spatial feature dimensions, and the multiple images in the database include this third image. Specifically, the proximity network can be based on... Figure 8 The method described in S802 of the illustrated embodiment is used to obtain the image. In one example, two similar images are a group photo of a first person and a second person. The first image has higher clarity than the second image, but the expression of the first person in the first image is significantly better than that of the first person in the second image. In this case, the first device can retain the first image and automatically delete the second image.

[0293] For example, Figure 11 A schematic flowchart of another image organization method 1100 provided in an embodiment of this application is shown. Figure 11 As shown, the method 1100 includes:

[0294] S1101: Obtain information about each image in at least one feature dimension from multiple images in the first album. The information about each image in at least one feature dimension includes information about each image in the semantic feature dimension. The information about each image in the semantic feature dimension includes information obtained by semantic analysis of the content of each image.

[0295] The information in the semantic feature dimension of each image includes information obtained by analyzing each image. For example, it may include information obtained by semantic analysis of the overall graphics and / or graphics segmentation results in each image, as well as information obtained by semantic analysis of the text in each image.

[0296] In some embodiments, the information of an image in the semantic feature dimension may include one or more of the following: a description of the foreground of the image, a description of the background of the image, a description of the relationship between objects in the image, a description of objects in the image, and an overview of the image content.

[0297] The description of the background of an image may include descriptions of people, objects, or the environment in the background, as well as descriptions of the relationships between them. For example, the background of an image may contain several children playing on the beach, a sunny and calm day, or a stormy and rainy day.

[0298] The description of the relationship between objects in the image is as follows: if the image includes a person and a cat, the relationship between the objects in the image could be, for example, a person holding a cat; if the image includes an adult and a child, the relationship between the objects in the image could be, for example, an adult holding a child. The objects mentioned here can be objects in the foreground or objects in the background.

[0299] The description of the object in the image may include one or more of the following: description of the object's behavior, description of the object's appearance, description of the object's expression, description of the object's clothing. In addition, it may also be other descriptions of the object, which are not limited in this application.

[0300] S1102: Generate the title of the first album based on information of each image in at least one feature dimension.

[0301] In some embodiments, a title for the first album can be generated based on information about each image in at least one feature dimension and an intimacy network. The intimacy network includes information indicating the degree of intimacy between a first object and one or more other objects, and / or information indicating the degree of intimacy between a second object and one or more other objects, wherein the first object, the second object, and the other objects are objects appearing in at least one image in the first album.

[0302] The proximity network can be, for example, derived from clustering multiple images in a photo library based on information in at least one feature dimension. This at least one feature dimension includes one or more of semantic, visual, temporal, and spatial feature dimensions, and the multiple images in the photo library include images from a first album. Specifically, it can be based on... Figure 8 The method described in S802 of the illustrated embodiment is used to obtain the solution.

[0303] The intimacy network can also be generated based on the clustering of images in the first album based on information in at least one feature dimension.

[0304] It is understandable that the above... Figures 8 to 11 The embodiments shown can be implemented individually or in combination of any two or more embodiments.

[0305] One or more modules or units described herein can be implemented in software, hardware, or a combination of both. When any of the above modules or units are implemented in software, the software exists as computer program instructions and is stored in memory. A processor can be used to execute the program instructions and implement the above method flow. The processor can include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., and various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor can be built into a SoC (System-on-a-Chip) or an application-specific integrated circuit (ASIC), or it can be a separate semiconductor chip. In addition to the cores within the processor for executing software instructions to perform calculations or processing, it may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0306] When the modules or units described herein are implemented in hardware, the hardware may be any one or any combination of a CPU, microprocessor, DSP, MCU, artificial intelligence processor, ASIC, SoC, FPGA, PLD, application-specific digital circuit, hardware accelerator, or non-integrated discrete device, which may run the necessary software or perform the above method flow independently of software.

[0307] When the modules or units described herein are implemented using software, they can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0308] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0309] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0310] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0311] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0312] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0313] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0314] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for organizing an image library, characterized in that, The method is applied to a first device, and the method includes: The information of each image in a library is obtained in at least one feature dimension, where the at least one feature dimension includes a semantic feature dimension, and the information of each image in the semantic feature dimension includes information obtained by semantic analysis of the content of each image. Based on the information of each image in at least one feature dimension, the multiple images are clustered to generate one or more recommended albums.

2. The method according to claim 1, characterized in that, For any one of the plurality of images, the information of the image in the semantic feature dimension includes one or more of the following: a description of the foreground of the image, a description of the background of the image, a description of the relationship between objects in the image, a description of objects in the image, and an overview of the content of the image.

3. The method according to claim 2, characterized in that, The description of the object in the image includes one or more of the following: a description of the object's behavior, a description of the object's appearance, a description of the object's expression, and a description of the object's clothing.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: By analyzing the clustering results of the multiple images, a proximity network corresponding to the multiple images is constructed. The proximity network is used to indicate the degree of proximity between a first object and one or more other objects and / or the degree of proximity between a second object and one or more other objects, wherein the first object, the second object, and the other objects are objects that appear in at least one of the multiple images.

5. The method according to claim 4, characterized in that, The first object is the user of the first device.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The title of the first recommended album is generated based on the information of each image in the first recommended album in the semantic feature dimension.

7. The method according to claim 4, characterized in that, The method further includes: The title of at least one of the recommended albums is generated based on the intimacy network.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The first device displays a first interface, which includes at least one of the one or more recommended albums.

9. The method according to claim 8, characterized in that, The at least one recommended album may be multiple recommended albums. The display order of the multiple recommended albums on the first interface is determined according to the image shooting time, or the display order of the multiple recommended albums on the first interface is determined according to the user's degree of interest in the event corresponding to the recommended album. The degree of interest is obtained by analyzing the clustering results of the multiple images.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: In response to a user's operation on a first recommended album in the at least one recommended album, the first device displays a second interface, which includes multiple sub-albums belonging to the first recommended album, and the multiple sub-albums correspond to multiple subcategories.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: When a new image is added to the image library, if the similarity between the information of the first image in the semantic feature dimension and the information of the images in the first recommended album in the semantic feature dimension of one or more recommended albums is greater than a preset similarity, then the first image is added to the first recommended album.

12. The method according to any one of claims 1 to 11, characterized in that, The method further includes: In response to a user selecting one or more images from the gallery, a second album is generated; In response to the user's operation of viewing the second album, the first device displays a third interface, which includes one or more images selected by the user, and one or more candidate images, wherein the one or more candidate images are obtained from images in the gallery that have not been selected by the user, and the similarity between the information of the one or more candidate images in at least one feature dimension and the information of the one or more images selected by the user in at least one feature dimension is greater than a preset similarity. In response to a user's action of selecting one or more target images from the one or more candidate images, the one or more target images are added to the second album.

13. The method according to claim 12, characterized in that, The method further includes: When a second image is added to the gallery, if the similarity between the information of the second image in the semantic feature dimension and the information of the image in the second album in the semantic feature dimension is greater than a preset similarity, then the second image is added to the second album.

14. The method according to any one of claims 1 to 13, characterized in that, The at least one feature dimension also includes one or more of the following: visual feature dimension, temporal feature dimension, and spatial feature dimension.

15. The method according to any one of claims 1 to 14, characterized in that, The image library includes third images, and the method further includes: Whether the third image is invalid is determined based on the user's behavior related to the content of the third image and / or the expiration time determined based on the content of the third image; When the third image is determined to be invalid, it will be moved to the Clean Up album.

16. A method for organizing an image library, characterized in that, The method is applied to a first device, and the method includes: The information of each image in a first album is obtained in at least one feature dimension, the at least one feature dimension including a semantic feature dimension, and the information of each image in the semantic feature dimension includes information obtained by semantic analysis of the content of each image. The title of the first album is generated based on information from each image in at least one feature dimension.

17. The method according to claim 16, characterized in that, The step of generating a title for the first album based on information from each image in at least one feature dimension includes: The title of the first album is generated based on information from each image in at least one feature dimension and an intimacy network, the intimacy network including information indicating the degree of intimacy between a first object and one or more other objects and / or information indicating the degree of intimacy between a second object and one or more other objects, wherein the first object, the second object, and the other objects are objects appearing in at least one image in the first album.

18. The method according to claim 16 or 17, characterized in that, For any one of the plurality of images, the information of the image in the semantic feature dimension includes one or more of the following: a description of the foreground of the image, a description of the background of the image, a description of the relationship between objects in the image, a description of objects in the image, and an overview of the content of the image.

19. An electronic device, characterized in that, include: One or more processors; One or more memory units; And one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method as claimed in any one of claims 1 to 15, or to perform the method as claimed in any one of claims 16 to 18.

20. A computer-readable storage medium, characterized in that, The storage medium stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 15, or the method as described in any one of claims 16 to 18.

21. A chip, characterized in that, The chip stores instructions that, when executed, implement the method as described in any one of claims 1 to 15, or the method as described in any one of claims 16 to 18.

22. A computer program product, characterized in that, The computer program product stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 15, or the method as described in any one of claims 16 to 18.

Images