Photo processing method and related equipment

Abstract

The embodiment of the invention provides a photo processing method and related equipment, and the method comprises the steps: obtaining a plurality of first photos which are not contained in a photo set when electronic equipment is in a preset state; dividing the plurality of first photos into a plurality of photo sets; if the pictures in any one of the plurality of picture sets are not analyzed, identifying the types of the pictures which are not analyzed; and if all the unanalyzed photos are successfully analyzed, determining a theme matched with each photo according to the type of each photo in the plurality of photo sets, and adding the photos to the photo sets corresponding to the themes. According to the embodiment of the invention, when the plurality of obtained photos comprise the photos which are not analyzed, the photos which are not analyzed can be analyzed in time, so that the corresponding photo set can be generated according to the types of all the photos, and the generation efficiency of the photo set is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, which belongs to image processing technology, and in particular to a photo processing method and related equipment. Background Art

[0002] With the development of terminal technology, smart terminal devices such as smartphones and tablets with shooting functions allow users to record various scenes by taking photos or videos. As the use time of smart terminal devices increases, more and more photos are taken and stored. In order to facilitate users to view different types of photos, the photos can be analyzed and stored in different photo collections according to the subject classification of the photos. However, in some scenarios where a large number of photos may be received, for example, when a user goes out to play, he may take a large number of photos in a short time, or receive a large number of photos transferred from other devices. The smart terminal device needs to process a large number of photos, resulting in a long time for photo processing and low efficiency in generating photo collections. Summary of the invention

[0003] In view of the above, it is necessary to provide a photo processing method and related equipment to solve the problem that there are too many photos in the electronic device and the user needs to spend a lot of time to organize the photos.

[0004] In a first aspect, the present application provides a photo processing method, which is applied to an electronic device, and the method includes: when the electronic device is in a preset state, obtaining multiple first photos that are not included in a photo collection; dividing the multiple first photos into multiple photo sets; if the photos in any photo set of the multiple photo sets have not completed the analysis, identifying the type of the photos that have not completed the analysis; if all the photos that have not completed the analysis are analyzed successfully, determining the theme that matches each photo according to the type of each photo in the multiple photo sets, and adding the photos to the photo collection corresponding to the theme.

[0005] Through the above technical solution, when the electronic device is in a state suitable for photo analysis, multiple photos that are not included in the photo collection are obtained, and then it is determined whether all the obtained photos have completed the analysis. If any photo has not completed the analysis, the unfinished photo is analyzed in time to identify the photo type. After the unfinished photos have been analyzed successfully, the types of all the obtained photos match the subjects of the photos, and the photos are added to the photo collection corresponding to the subject, ensuring that the photos not included in the photo collection can be analyzed and processed in time, and then the photos are added to the photo collection corresponding to the subject in time, which effectively reduces the photo processing time and improves the generation efficiency of the photo collection.

[0006] In a possible implementation, the method further includes: if the analysis of all photos in the photo collection has been completed, determining a theme matching each photo according to the type of each photo in the multiple photo collections, and adding the photo to the photo collection corresponding to the theme.

[0007] Through the above technical solution, after determining that all photos in the photo collection have been analyzed, the photos are promptly added to the photo collection corresponding to the theme, which effectively reduces the photo processing time and improves the efficiency of photo collection generation.

[0008] In a possible implementation, the method further includes: if analysis of any of the unanalyzed photos fails, determining that generation of the photo collection corresponding to the multiple first photos fails.

[0009] Through the above technical solution, after determining that the photo collection corresponding to the multiple first photos fails to be generated, the corresponding response method can be determined in time to ensure that the photo collection corresponding to the multiple first photos is successfully generated.

[0010] In a possible implementation, the preset state includes the electronic device charging with the screen turned off, the power of the electronic device being greater than or equal to a preset power, and the temperature of the electronic device being less than or equal to a preset temperature.

[0011] Through the above technical solution, photo analysis can be performed when the electronic device is in a state where the screen is off, charging, the battery is high, and the temperature is low, that is, in a state suitable for photo analysis, to avoid the photo analysis affecting the normal use of the electronic device.

[0012] In one possible implementation, when the electronic device is in a preset state, multiple first photos that are not included in the photo collection are obtained, including: communicating with the power management service, display management service and temperature sensor of the electronic device through a status listener to monitor the screen on and off status, charging status, power level and temperature of the electronic device; if the status listener determines that the electronic device is in the preset state, a generation instruction for a photo collection is generated, and in response to the generation instruction for the photo collection, multiple first photos that are not included in the photo collection are obtained from the gallery database of the electronic device.

[0013] Through the above technical solution, the state of the electronic device can be monitored by the state monitor, so that the electronic device can be monitored in a timely manner to be in a preset state.

[0014] In a possible implementation, when the electronic device is in a preset state, obtaining multiple first photos that are not included in a photo collection includes: when the electronic device is in the preset state, determining whether a gallery application of the electronic device includes a photo collection; if the gallery application does not include a photo collection, obtaining all photos in the gallery database; if the gallery application includes a photo collection, obtaining photos in the gallery database that are not stored in a folder corresponding to the photo collection.

[0015] Through the above technical solution, the photos not included in the photo collection can be determined according to whether the photo collection is included in the gallery application, so as to accurately obtain multiple first photos.

[0016] In a possible implementation, when the electronic device is in a preset state, obtaining multiple first photos that are not included in the photo collection includes: determining whether the photo information of each photo in the gallery database includes photo collection information; if the photo information of the photo includes the photo collection information, determining that the photo is included in the photo collection; if the photo information of the photo does not include the photo collection information, determining that the photo is not included in the photo collection.

[0017] Through the above technical solution, photos not included in the photo collection can be determined according to the photo collection information in the gallery database, so as to accurately obtain multiple first photos.

[0018] In a possible implementation manner, dividing the plurality of first photos into a plurality of photo sets includes: clustering the plurality of first photos obtained to obtain a plurality of second photos corresponding to a plurality of cluster categories.

[0019] According to the above technical solution, by clustering multiple first photos to obtain multiple second photos corresponding to multiple clustering categories, the multiple first photos can be divided into multiple photo sets for analysis and processing, which effectively improves the analysis and processing efficiency of the photos.

[0020] In a possible implementation, clustering the acquired multiple first photos to obtain multiple second photos corresponding to multiple cluster categories includes: time clustering the multiple first photos to obtain multiple second photos corresponding to multiple preset time ranges.

[0021] According to the above technical solution, by performing time clustering on multiple first photos to obtain multiple second photos corresponding to multiple preset time ranges, the multiple first photos can be divided into multiple photo sets for analysis and processing, which effectively improves the analysis and processing efficiency of the photos.

[0022] In a possible implementation, clustering the acquired multiple first photos to obtain multiple second photos corresponding to multiple cluster categories includes: clustering the multiple first photos by location to obtain multiple second photos corresponding to multiple location ranges.

[0023] According to the above technical solution, by performing position clustering on multiple first photos to obtain multiple second photos corresponding to multiple position ranges, the multiple first photos can be divided into multiple photo sets for analysis and processing, which effectively improves the analysis and processing efficiency of the photos.

[0024] In one possible implementation, the identifying the type of the photo for which analysis has not been completed includes: preprocessing the photo for which analysis has not been completed and inputting the preprocessing into a target detection model; extracting a feature map of the photo for which analysis has not been completed through the target detection model; extracting a region of interest of the photo for which analysis has not been completed based on the feature map of the photo for which analysis has not been completed through the target detection model; extracting a feature map of the region of interest from the feature map of the photo for which analysis has not been completed based on the region of interest of the photo for which analysis has not been completed through the target detection model; and classifying the region of interest based on the features of the region of interest of the photo for which analysis has not been completed through the target detection model to determine the type of the target object in the photo for which analysis has not been completed.

[0025] According to the above technical solution, the type of the target object in the photo is accurately identified through the target detection model, thereby accurately determining the type of the photo.

[0026] In a possible implementation, the method further includes: during the process of analyzing the photos in the multiple photo sets, if the electronic device is not in the preset state, pausing the analysis of the photos in the multiple photo sets; if the electronic device is in the preset state again, obtaining a photo set containing photos that failed to be analyzed, analyzing each photo that failed to be analyzed in the obtained photo set, and identifying the type of each photo that failed to be analyzed.

[0027] According to the above technical solution, when the electronic device is in the preset state again, the photos that failed to be analyzed are analyzed and processed first, ensuring that the photos not included in the photo collection can be analyzed and processed in time, and then the photos are added to the photo collection corresponding to the theme in time, which effectively reduces the photo processing time and improves the efficiency of generating photo collections.

[0028] In a second aspect, the present application provides an electronic device, comprising a memory and a processor: wherein the memory is used to store program instructions; the processor is used to read and execute the program instructions stored in the memory, and when the program instructions are executed by the processor, the electronic device performs the above-mentioned photo processing method.

[0029] In a third aspect, the present application provides a chip coupled to a memory in an electronic device, wherein the chip is used to control a processor of the electronic device to execute the above-mentioned photo processing method.

[0030] In a fourth aspect, the present application provides a computer storage medium storing program instructions, which, when executed on an electronic device, enables the electronic device to execute the above-mentioned photo processing method.

[0031] In addition, the technical effects brought about by the second to fourth aspects can be found in the descriptions related to the methods of each design in the above method part, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of a gallery application program interface provided in an embodiment of the present application.

[0033] Figure 2 It is a timing diagram of a photo processing method provided in one embodiment of the present application.

[0034] Figure 3 It is a software architecture diagram of an electronic device provided in one embodiment of the present application.

[0035] Figure 4 It is a flowchart of a photo processing method provided in one embodiment of the present application.

[0036] Figure 5 This is a flowchart for analyzing photos provided by an embodiment of the present application.

[0037] Figure 6 It is a schematic diagram of the network structure of the target detection model provided in one embodiment of the present application.

[0038] Figure 7 This is a flowchart of a photo processing method provided by another embodiment of the present application.

[0039] Figure 8 This is a flowchart of a photo processing method provided by another embodiment of the present application.

[0040] Fig. 9 This is a partial timing diagram of a photo processing method provided in one embodiment of the present application.

[0041] Fig.10This is a partial timing diagram of a photo processing method provided in one embodiment of the present application.

[0042] Fig.11 This is a hardware architecture diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0043] The terms "first" and "second" involved in an embodiment of the present application are only used for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in an embodiment of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by technicians in the technical field in this application. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. It should be understood that, unless otherwise specified in this application, " / " means or. For example, A / B can represent A or B. "And / or" in this application is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. "At least one" refers to one or more. "Multiple" refers to two or more than two. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c. In the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0045] For the detailed implementation process of the photo processing method, please refer to the description in each embodiment below.

[0046] In order to better understand the photo processing method provided in the embodiment of the present application, Figure 1 and Figure 2 The application scenarios of the photo processing method provided in the embodiments of the present application are described.

[0047] With the development of terminal technology, smart terminal devices such as smartphones and tablet computers with shooting functions allow users to record various scenes by taking photos or videos. As the use time of smart terminal devices increases, more and more photos are taken and stored. In order to facilitate users to view photos according to different types or themes, photos can be analyzed and identified to determine the type or theme of the photos, and different photo collections can be generated according to the type or theme of the photos. In this way, multiple photos of the same type or theme can be viewed by viewing the photo collection.

[0048] See also Figure 1 As shown, it is a schematic diagram of the gallery application interface provided by an embodiment of the present application. The photos displayed on the gallery application (also referred to as the album application) interface are stored in the gallery database, which is a storage area in the memory of the electronic device for storing photos and / or videos. The gallery application can create multiple folders in the gallery database according to different types or themes (such as portraits, landscapes, food, pets, etc.), each folder is used to store a collection of photos of the corresponding type or theme, and the photo collection is presented in the form of an album in the gallery application. The types or themes of the photo collection include, but are not limited to, portraits, pets, food, and scenery. The portrait photo collection includes photos containing portraits in the gallery database, the pet photo collection includes photos containing pets in the gallery database, the food photo collection includes photos containing food in the gallery database, and the landscape photo collection includes photos containing natural landscapes in the gallery database. By dividing the photos in the gallery database into different photo collections according to different types or themes, it is convenient for users to view multiple photos of different types or different themes without having to flip through all the photos.

[0049] However, in some scenarios where electronic devices receive a large number of photos in a short period of time, for example, when a user takes a large number of photos in a short period of time while out and about, or when an electronic device receives a large number of photos transferred from other devices via a wireless network (such as Bluetooth or Wi-Fi, etc.), the electronic device needs to analyze and process a large number of photos to create different photo collections, or add photos to already created photo collections. When there are a large number of photos to be processed, the electronic device takes a long time to process the photos, resulting in a low efficiency in generating photo collections. In addition, due to factors such as low photo quality, inability to extract photo features, and paused photo processing, photo processing may be abnormal, resulting in analysis and processing failure, that is, the type or subject of the photo cannot be identified, resulting in the inability to generate the photo collection normally.

[0050] See also Figure 2The figure shows a processing sequence diagram of the photo processing method provided by an embodiment of the present application. The status of the electronic device is monitored by the status listener of the electronic device. When the status listener detects that the electronic device is in the charging and screen-off state, the moment service of the gallery application is triggered. The moment service queries and obtains the photos to be analyzed in the gallery database. The gallery database returns the photo information to the moment service. The moment service verifies whether the acquired photos have completed the label analysis. The label analysis is to analyze the type of the photo and use the type of the photo as the label of the photo. If the acquired photos have completed the label analysis, the subject corresponding to the label is matched, and a photo collection corresponding to the subject is generated. However, if any photo has not completed the label analysis, the acquired photo cannot be added to the photo collection, that is, a photo collection cannot be generated based on the acquired photos.

[0051] In addition, electronic devices consume high power when processing a large number of photos, which may affect the processing of other application processes, causing the electronic devices to freeze and run less smoothly.

[0052] In order to reduce photo processing time, improve the efficiency of generating photo collections, and avoid photo processing affecting other application processes, an embodiment of the present application provides a photo processing method, which can analyze and process photos when the electronic device is in a low power consumption state, and promptly analyze photos that have not been completed or failed to be analyzed, to ensure that the photo collection corresponding to the photo can be generated normally.

[0053] See also Figure 3 As shown, it is a software architecture diagram of the electronic device provided in the embodiment of the present application. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. For example, the Android system is divided into four layers, from top to bottom, respectively, the application layer 101, the framework layer 102, the Android runtime (Androidruntime) and the system library 103, the hardware abstraction layer 104, the kernel layer 105, and the hardware layer 106.

[0054] The application layer 101 may include a series of application packages, such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, device control service and other applications.

[0055] The framework layer 102 provides an application programming interface (API) and a programming framework for the application programs of the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0056] Among them, the window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to the application. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc. The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying pictures. The phone manager is used to provide communication functions for electronic devices. For example, the management of call status (including connected, hung up, etc.). The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, etc. The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloads, message reminders, etc. The notification manager can also be a notification that appears in the top status bar of the system in the form of an icon or scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0057] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. The core library contains two parts: one is the function that the Java language needs to call, and the other is the Android core library.

[0058] The application layer 101 and the framework layer 102 run in a virtual machine. The virtual machine executes the Java files of the application layer and the framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0059] The system library 103 may include multiple functional modules, such as a surface manager, a media library, a 3D graphics processing library (eg, OpenGL ES), a 2D graphics engine (eg, SGL), and the like.

[0060] Among them, the surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple commonly used audio and video formats, as well as static image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing.

[0061] The hardware abstraction layer 104 runs in the user space, encapsulates the kernel layer driver, and provides a calling interface to the upper layer.

[0062] The kernel layer 105 is a layer between hardware and software and includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0063] The kernel layer 105 is the core of the operating system of the electronic device. It is the first layer of software expansion based on hardware, provides the most basic functions of the operating system, is the basis of the operating system, is responsible for managing the system's processes, memory, device drivers, files and network systems, and determines the performance and stability of the system. For example, the kernel can determine the operating time of an application on a certain part of the hardware.

[0064] The kernel layer 105 includes programs closely related to the hardware, such as interrupt handlers, device drivers, etc., and also includes basic, common, and high-frequency modules, such as clock management modules, process scheduling modules, etc., and also includes key data structures. The kernel layer can be set in the processor or solidified in the internal memory.

[0065] The hardware layer 106 includes the hardware of the electronic device, such as a display screen, buttons, a camera, etc.

[0066] See also Figure 4 FIG. 1 is a flowchart of a photo processing method provided by an embodiment of the present application. The method is applied in an electronic device, and the photo processing method includes:

[0067] S101, when the electronic device is in a preset state, obtaining a plurality of first photos that are not included in a photo collection.

[0068] In one embodiment of the present application, the preset state includes the electronic device charging with the screen off, the power of the electronic device is greater than or equal to the preset power, and the temperature of the electronic device is less than or equal to the preset temperature. The preset power is a percentage of power, for example, 50%, 60%, 70% or other percentage values, and the preset temperature is 30°C, 25°C, 20°C or other temperature values. The above values ​​are only for illustration and are not limited to this in practical applications. The embodiment of the present application does not limit the specific content of the preset state. In other embodiments of the present application, the preset state may also include the electronic device charging with the screen off and the power of the electronic device is greater than or equal to the preset power, or the preset state may also include the electronic device charging with the screen off and the electronic device connected to a wireless local area network (Wireless Local Area Network, WLAN), or the preset state may only include the electronic device charging with the screen off.

[0069] In one embodiment of the present application, the status listener is a function for monitoring the status of an electronic device. For example, the status of an electronic device may include battery power, temperature, network connection status, memory usage, etc. The status listener may communicate with the power management service, display management service, and temperature sensor of the operating system to monitor the status parameters of the electronic device such as the screen on and off status, charging status, power level, and temperature. For example, if the status listener determines that the electronic device is in a state of charging with the screen off, the power level is greater than or equal to the preset power level, and the temperature is less than or equal to the preset temperature, a generation instruction for a photo collection is generated, and the moment service responds to the generation instruction for the photo collection to obtain multiple first photos that are not included in the photo collection from the gallery database. In other embodiments of the present application, the status listener may also be a tool for monitoring the status of a device.

[0070] In one embodiment of the present application, when the electronic device is in a preset state, it is determined whether the gallery application of the electronic device includes a photo collection. If the gallery application does not include a photo collection, all photos in the gallery database are not included in the photo collection, and therefore, all photos in the gallery database can be obtained. If the gallery application includes a photo collection, photos in the gallery database that are not stored in a folder corresponding to the photo collection are photos that are not included in the photo collection, and therefore, photos that are not stored in a folder corresponding to the photo collection can be obtained from the gallery database.

[0071] In another embodiment of the present application, the gallery database stores the photo identification and photo information of the photo. If the photo is included in a photo collection, the photo information of the photo includes the photo collection information of the photo collection in which the photo is located. For example, the photo collection information may be the serial number of the photo collection or the theme name of the photo collection. It can be determined whether the photo information of each photo in the gallery database includes the photo collection information. If the photo information of the photo includes the photo collection information, it is determined that the photo is included in the photo collection. If the photo information of the photo does not include the photo collection information, it is determined that the photo is not included in the photo collection. Therefore, multiple first photos that are not included in the photo collection can be obtained by obtaining photos whose photo information does not include the photo collection information from the gallery database.

[0072] S102: Divide the plurality of first photos into a plurality of photo sets.

[0073] In one embodiment of the present application, the multiple first photos obtained are taken as an event set, and the event set includes an event of adding each first photo obtained to an existing or newly created photo collection. The multiple first photos obtained are clustered to obtain multiple second photos corresponding to multiple cluster categories, and the multiple second photos of each cluster category are taken as sub-event sets, each sub-event set is a photo set, so that the multiple first photos are divided into multiple photo sets, and the sub-event set includes an event of adding multiple second photos of each cluster category to an existing or newly created photo collection. Each sub-event set, that is, each photo set has a photo set identifier (ID).

[0074] In one embodiment of the present application, multiple first photos can be time-clustered to obtain multiple second photos corresponding to multiple preset time ranges, thereby generating multiple photo sets. The preset time range can be a day, a week, a month, or other time ranges. Determine the photos among the multiple first photos whose shooting time is within a preset time range before the current time (for example, the first month before the current time), determine the photos among the multiple first photos whose shooting time is within another preset time range before the current time (for example, the second month before the current time), and so on, until all the first photos are time-clustered to obtain multiple second photos corresponding to multiple preset time ranges, and take the multiple second photos corresponding to each preset time range as a photo set, thereby generating multiple photo sets.

[0075] In another embodiment of the present application, multiple first photos may be clustered by location to obtain multiple second photos corresponding to multiple location ranges, thereby generating multiple photo sets. The location information of each first photo, such as longitude and latitude information, may be obtained, and the first photos belonging to the same location range may be used as a clustering category to obtain multiple second photos corresponding to multiple location ranges. The location range may be a preset distance range or a preset geographical location, for example, the preset distance range may be a 50-kilometer range, and the preset geographical location may be Shenzhen City, Futian District, etc.

[0076] S103, determining whether all the photos in the photo set have been analyzed. If all the photos in the photo set have been analyzed, executing S106; if any photo in any photo set has not been analyzed, executing S104.

[0077] In one embodiment of the present application, if the photo information of each photo in the photo collection includes the photo type, the photo is searched in the photo library database according to the photo identification of the photo in the photo collection. If the photo information of the photo includes the photo type, it is determined that the photo has been analyzed. If the photo information of the photo does not include the photo type, it is determined that the photo has not been analyzed. Therefore, if the photo information of the photos in all photo collections includes the photo type, it is determined that the photos in all photo collections have been analyzed. If the photo information of any photo in any photo collection does not include the photo type, it is determined that the photos in any photo collection have not been analyzed.

[0078] S104, identifying the type of the photo that has not been analyzed.

[0079] In one embodiment of the present application, a photo set including photos that have not been analyzed (hereinafter referred to as incomplete analysis photos) is obtained in a gallery database according to a photo set identifier, and each incomplete analysis photo in the photo set is analyzed one by one by a target detection model to identify the type of the target object in the incomplete analysis photo, and the type of the target object is used as the photo type, and the photo type is stored in the photo information of the corresponding photo in the gallery database according to the photo identifier. The photos in the photo set can be input into the target detection model, and the features of the target objects of the photos can be extracted by the target detection model, and the target objects can be classified according to the features of the target objects, and the types of the target objects in the photos can be output. Among them, the target detection model can be a fast regional convolutional neural network model (Faster Region-based Convolutional Neural Networks, faster R-CNN), a YOLO target detection model, etc.

[0080] S105, judging whether all the photos that have not been analyzed have been analyzed successfully. If all the photos that have not been analyzed have been analyzed successfully, executing S106; if any of the photos that have not been analyzed failed to be analyzed, executing S107.

[0081] In one embodiment of the present application, the photo information of the unfinished analysis is searched in the gallery database according to the photo identification of the unfinished analysis photo in each photo set to see whether the photo type is included. If the photo information of the unfinished analysis photo includes the photo type, it is determined that the unfinished analysis photo has been analyzed successfully. If the photo information of the unfinished analysis photo still does not include the photo type, it is determined that the analysis of the unfinished analysis photo has failed.

[0082] It is understandable that when the electronic device changes from being in the preset state to not being in the preset state, the analysis and processing of the photo is suspended, and the photo being analyzed and processed cannot identify the photo type, which is a photo that fails to be analyzed. Alternatively, the photo may fail to be analyzed due to low quality.

[0083] S106, determining the theme matching the photos in each photo collection according to the photo type, and adding the photos to the photo collection corresponding to the theme.

[0084] In one embodiment of the present application, the gallery application of the electronic device includes a plurality of preset themes, and the plurality of preset themes may include themes of photos frequently taken by the user, for example, the plurality of preset themes include but are not limited to portraits, food, scenery, pets, and architecture. Each preset theme includes a plurality of preset object types, and it is determined whether the type of the target object in the photo is the preset object type. If the type of the target object in the photo is the preset object type, the preset theme corresponding to the preset object type is determined as the theme that matches the photo. For example, the preset object type of the portrait theme is a face. If the type of the target object of the photo is identified as a face, which is the preset object type, then the theme that matches the photo is determined to be a portrait. For another example, the preset object type of the food theme is a cake. If the type of the target object of the photo is identified as a cake, which is the preset object type, then the theme that matches the photo is determined to be food.

[0085] In one embodiment of the present application, if the photo collection corresponding to the theme is included in the gallery application, the photo is stored in a folder corresponding to the photo collection corresponding to the theme, thereby adding the photo to the photo collection corresponding to the theme. If the photo collection corresponding to the theme is not included in the gallery application, a photo collection corresponding to the theme is created, and the photo is stored in a folder corresponding to the photo collection corresponding to the theme, thereby adding the photo to the photo collection corresponding to the theme.

[0086] S107, determining that generation of a photo collection corresponding to the plurality of first photos fails.

[0087] In one embodiment of the present application, if the analysis of photos in any photo collection fails, that is, the target detection model does not identify the type of the target object in any photo, it is determined that the generation of the photo collection of multiple first photos has failed, and the analysis results of each first photo and the photo collection identifier containing the photos that failed the analysis are stored in the gallery database.

[0088] Through the above embodiment, when the electronic device is in a state suitable for photo analysis, multiple photos that are not included in the photo collection are obtained, and then it is determined whether all the obtained photos have completed the analysis. If any photo has not completed the analysis, the unfinished photo can be analyzed to identify the photo type, and then it is determined whether the unfinished analysis photos have been successfully analyzed. After the unfinished analysis photos have been successfully analyzed, all the obtained photos have corresponding types, and the themes of the photos are matched according to the photo types, and the photos are added to the photo collection corresponding to the theme. The unfinished analysis photos can be analyzed and processed in time to ensure that the photos not included in the photo collection can be analyzed and processed in time, so as to determine the photo type and match the photo theme, so as to add the photos to the photo collection corresponding to the theme in time, effectively reducing the photo processing time and improving the generation efficiency of the photo collection.

[0089] See also Figure 5 As shown, it is a flowchart of analyzing photos provided by an embodiment of the present application.

[0090] S1041, pre-process the photos in each photo set and input them into the target detection model.

[0091] In one embodiment of the present application, the photos in each photo set are unfinished analysis photos in the photo set including unfinished analysis photos, and the detailed process of analyzing the photos is described based on the example of the target detection model being a fast regional convolutional neural network model. The photos are converted to a preset size, the photos of the preset size are converted to grayscale images, and the grayscale images corresponding to the photos are used as input data and input into the target detection model. The preset size is the resolution, for example, it can be 800*600, or other suitable sizes.

[0092] S1042, extracting a feature map of the photo through an object detection model.

[0093] See also Figure 6As shown, it is a schematic diagram of the network structure of the target detection model provided by an embodiment of the present application. The target detection model 20 includes a feature extraction network 21, a region candidate network 22, an interest region pooling network 23 and a classification network 24. The target detection model receives a grayscale image corresponding to the input photo, and extracts a feature map of the photo through the feature extraction network. The feature extraction network includes 13 convolutional layers, 13 activation function layers and 4 pooling layers. The convolution kernel size of the convolution layer is 3, the number of padded pixels is 1, and the convolution step size is 1. The convolution kernel size of the pooling layer is 2, the number of padded pixels is 0, and the convolution step size is 2.

[0094] S1043, extracting the region of interest of the photo through the target detection model according to the feature map of the photo.

[0095] In one embodiment of the present application, the feature extraction network transmits the extracted feature map to the region candidate network, the region candidate network performs convolution processing on the feature map, determines the offset of the candidate region of interest and the region of interest regression based on the feature map after the convolution processing, and determines the region of interest based on the offset of the candidate region of interest and the region of interest regression.

[0096] S1044, according to the region of interest of the photo, extracting a feature map of the region of interest from the feature map of the photo through a target detection model.

[0097] In one embodiment of the present application, the region candidate network transmits the region of interest to the region of interest pooling network, the feature extraction network transmits the extracted feature map to the region of interest pooling network, the region of interest pooling network converts the sizes of multiple regions of interest to the same size as the feature map of the photo, extracts the feature map corresponding to each region of interest on the feature map of the photo, performs maximum pooling processing on the feature map corresponding to each region of interest on the feature map of the photo, and obtains the feature map of each region of interest.

[0098] S1045, classifying the interest region according to the characteristics of the interest region of the photo through the target detection model, determining the type of the target object in the photo, and determining the type of the target object as the photo type.

[0099] In one embodiment of the present application, the region of interest pooling network transmits the feature map of each region of interest to the classification network, which includes a fully connected layer and a softmax (classification function) layer. The type of the region of interest is calculated through the fully connected layer and the softmax layer, the region of interest is classified, the type of the target object in the photo is determined, and the type of the target object is determined as the photo type.

[0100] Through the above embodiments, the target detection model can accurately identify the type of the target object in the photo, and then accurately determine the type of the photo.

[0101] See also Figure 7 FIG. 1 is a flowchart of a photo processing method provided by another embodiment of the present application. The method is applied in an electronic device, and the photo processing method includes:

[0102] S201, when the electronic device is in a preset state, obtaining a plurality of first photos that are not included in the photo collection.

[0103] S202: Divide the plurality of first photos into a plurality of photo sets.

[0104] S203, determining whether all the photos in the photo set have been analyzed. If all the photos in the photo set have been analyzed, executing S206; if any of the photos in the photo set has not been analyzed, executing S204.

[0105] S204, identifying the type of the photo that has not been analyzed.

[0106] S205, judging whether all the unfinished analysis photos have been analyzed successfully. If all the unfinished analysis photos have been analyzed successfully, executing S206; if any unfinished analysis photo fails to be analyzed, executing S207.

[0107] S206, determining the theme matching the photos in each photo collection according to the photo type, and adding the photos to the photo collection corresponding to the theme.

[0108] S207, determining that generation of a photo collection corresponding to the plurality of first photos fails.

[0109] The specific implementation of S201-S207 is the same as that of S101-S107, and will not be repeated here.

[0110] S208, when the electronic device is in the preset state again, a photo set including photos that failed to be analyzed (hereinafter referred to as analysis failed photos) is obtained, and the type of the analysis failed photos is identified. Then, the process returns to S205.

[0111] In one embodiment of the present application, Figure 5 The process shown identifies the type of failed photos, and the number of photo sets obtained is less than or equal to a preset number, for example, 3, 4, 5 or other preset values.

[0112] Through the above embodiment, photos that failed analysis can be analyzed again in time to determine the types of all photos and match the photo themes, so that the photos can be added to the photo collection corresponding to the theme in time, effectively reducing the photo processing time and improving the efficiency of photo collection generation.

[0113] See also Figure 8FIG. 1 is a flowchart of a photo processing method provided by another embodiment of the present application. The method is applied in an electronic device, and the photo processing method includes:

[0114] S301, when the electronic device is in a preset state, obtaining a plurality of first photos that are not included in a photo collection.

[0115] S302: Divide the plurality of first photos into a plurality of photo sets.

[0116] S303, determine whether all the photos in the photo set have been analyzed. If all the photos in the photo set have been analyzed, execute S306; if any of the photos in the photo set has not been analyzed, return to execute S304.

[0117] S304, identifying the type of the photo that has not been analyzed.

[0118] S305, when the electronic device is not in a preset state, suspending the analysis of the unfinished analyzed photos.

[0119] In one embodiment of the present application, when the electronic device changes from being in a not-preset state to being in a not-preset state, the conditions for photo analysis and photo collection generation are not met, and therefore, the analysis of the photos that have not been completed is suspended.

[0120] S306, when the electronic device is in the preset state again, it is determined whether all the photos that have not been analyzed have been analyzed successfully. If all the photos that have not been analyzed have been analyzed successfully, S308 is executed; if any of the photos that have not been analyzed fails to be analyzed, S307 is executed.

[0121] S307, obtaining a photo set including photos that failed to be analyzed, and identifying the types of the photos that failed to be analyzed. Then returning to S306.

[0122] In one embodiment of the present application, the number of acquired photo sets is less than or equal to a preset number, for example, the preset number is 3, 4, 5 or other preset values.

[0123] S308, determining the theme matching the photos in each photo collection according to the photo type, and adding the photos to the photo collection corresponding to the theme.

[0124] Through the above embodiment, when the electronic device enters the preset state again, the photos that failed to be analyzed are analyzed again first, and then the photos that have not completed the analysis are processed, so as to ensure that the types of all photos are determined and the photo themes are matched, so as to timely add the photos to the photo collection corresponding to the theme, effectively reducing the photo processing time and improving the efficiency of generating photo collections.

[0125] See also Figure 9-10, which is a schematic diagram of the processing sequence of the photo processing method provided by an embodiment of the present application. The application layer 101 includes a state listener 1011, a gallery time service 1012, a gallery database 1013 and an event analysis module 1014. The framework layer 102 includes a tag analysis algorithm 1021.

[0126] S401, the status monitor 1011 monitors that the electronic device is in a preset state, that is, it monitors that the electronic device is charging with the screen off, the power is greater than or equal to the preset power, and the temperature is less than or equal to the preset temperature.

[0127] S402, the status listener 1011 sends a photo collection generation instruction to the gallery moment service 1012.

[0128] S403, the gallery moment service 1012 searches the gallery database 1013 for photos that are not included in the photo collection.

[0129] S404, the gallery database 1013 returns the multiple photos that are not included in the photo collection to the gallery moment service 1012.

[0130] S405, the gallery moment service 1012 clusters the multiple photos to obtain multiple small event sets.

[0131] S406, the gallery moment service 1012 checks whether the analysis of all photos in the small event set has been completed.

[0132] S407, if all photos in the small event set are analyzed, the gallery moment service 1012 determines the theme that the photo tag matches, and adds the photo to the photo collection corresponding to the theme. The photo tag is the photo type.

[0133] S408, if the analysis of any photo in the small event set is not completed, the gallery moment service 1012 stores the verification result of the failed photo collection generation in the gallery database 1013.

[0134] S409, the status monitor 1011 detects that the electronic device is not in a preset state.

[0135] S410, the status listener 1011 sends a pause generation instruction of the photo collection to the gallery moment service 1012.

[0136] S411, the gallery moment service 1012 suspends the analysis of the photos in the small event set.

[0137] S412, the status monitor 1011 monitors that the electronic device is in a preset state.

[0138] S413, the status listener 1011 sends a photo collection generation instruction to the gallery moment service 1012.

[0139] S414, the gallery moment service 1012 searches the gallery database 1013 for small event sets containing photos that failed to be analyzed. The number of small event sets containing photos that failed to be analyzed is less than or equal to a preset number, such as 3.

[0140] S415 , the gallery database 1013 returns the small event set including the photos that failed analysis to the gallery moment service 1012 .

[0141] S416 , the gallery moment service 1012 sends the small event set including the photos that failed analysis to the event analysis module 1014 .

[0142] S417 , the event analysis module 1014 sends a photo analysis instruction to the tag analysis algorithm 1021 .

[0143] S418, the tag analysis algorithm 1021 analyzes the photos that failed to be analyzed in the small event set including the photos that failed to be analyzed, and identifies the types of the photos that failed to be analyzed.

[0144] S419 , the tag analysis algorithm 1021 returns the photo type to the event analysis module 1014 .

[0145] S420 , the event analysis module 1014 stores the photo type in the gallery database 1013 .

[0146] S421, the event analysis module 1014 sends a photo collection generation instruction to the gallery moment service 1012.

[0147] S422, the gallery moment service 1012 searches the gallery database 1013 for photos in the small event set to be added to the photo collection according to the identifier of the analyzed small event set.

[0148] S423, the gallery database 1013 returns the photos and photo types to be added to the photo collection in the small event collection to the gallery moment service 1012.

[0149] S424, the gallery moment service 1012 matches the corresponding theme according to the type of the photos to be added to the photo collection in the small event set, and adds the photos to the photo collection corresponding to the theme.

[0150] S425, the gallery moment service 1012 stores the photo collection into the gallery database 1013.

[0151] The present application also provides an electronic device 100. Fig.11As shown, the electronic device 100 can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a vehicle-mounted device, a smart home device and / or a smart city device. The embodiments of the present application do not impose any special restrictions on the specific type of the electronic device 100.

[0152] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0153] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0154] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0155] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0156] The processor 110 may also be provided with a memory for storing instructions and data. In one embodiment of the present application, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use instructions or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0157] In one embodiment of the present application, the processor 110 may include one or more interfaces. The interface may include an Inter-integrated Circuit (I2C) interface, an Inter-integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and / or a Universal Serial Bus (USB) interface, etc.

[0158] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In one embodiment of the present application, the processor 110 may include multiple groups of I2C buses. The processor 110 can be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 can be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface to realize the touch function of the electronic device 100.

[0159] The I2S interface can be used for audio communication. In one embodiment of the present application, the processor 110 may include multiple I2S buses. The processor 110 may be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In one embodiment of the present application, the audio module 170 may transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call via a Bluetooth headset.

[0160] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In one embodiment of the present application, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In one embodiment of the present application, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering a call via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0161] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In one embodiment of the present application, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In one embodiment of the present application, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0162] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In one embodiment of the present application, the processor 110 and the camera 193 communicate via the CSI interface to realize the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to realize the display function of the electronic device 100.

[0163] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In one embodiment of the present application, the GPIO interface can be used to connect the processor 110 with the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0164] The USB interface 130 is an interface that complies with the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices 100, such as AR devices, etc.

[0165] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0166] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device 100 through the power management module 141.

[0167] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0168] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0169] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0170] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (Low Noise Amplifier, LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In one embodiment of the present application, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In one embodiment of the present application, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0171] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In one embodiment of the present application, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be arranged in the same device as the mobile communication module 150 or other functional modules.

[0172] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0173] In one embodiment of the present application, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include Global System For Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a Global Positioning System (GPS), a Global Navigation Satellite System (GLONASS), a Beidou Navigation Satellite System (BDS), a Quasi-Zenith Satellite System (QZSS) and / or a Satellite Based Augmentation System (SBAS).

[0174] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for photo processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0175] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, Microled, Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In one embodiment of the present application, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0176] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0177] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In one embodiment of the present application, ISP can be set in camera 193.

[0178] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In one embodiment of the present application, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0179] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0180] The video codec is used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0181] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0182] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0183] Random access memory may include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, the fifth generation DDR SDRAM is generally referred to as DDR5 SDRAM), etc.;

[0184] Non-volatile memory may include disk storage devices and flash memory.

[0185] Flash memory can be divided into NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. according to the operating principle. It can be divided into single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc. according to the storage unit potential level. It can be divided into universal flash storage (UFS), embedded Multi Media Card (eMMC), etc. according to the storage specification.

[0186] The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data.

[0187] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 110 .

[0188] The external memory interface 120 can be used to connect to an external non-volatile memory to expand the storage capacity of the electronic device 100. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos are stored in the external non-volatile memory.

[0189] The internal memory 121 or the external memory interface 120 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 110. The one or more computer programs include multiple instructions. When the multiple instructions are executed by the processor 110, the screen display detection method executed on the electronic device 100 in the above embodiment can be implemented to implement the screen display detection function of the electronic device 100.

[0190] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0191] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In one embodiment of the present application, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0192] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0193] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0194] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0195] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm Open Mobile Terminal Platform (OMTP) standard interface or a Cellular Telecommunications Industry Association of the USA (CTIA) standard interface.

[0196] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0197] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0198] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0199] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In one embodiment of the present application, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100. An embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on the electronic device 100, the electronic device 100 executes the above-mentioned related method steps to implement the photo processing method in the above-mentioned embodiment.

[0200] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the photo processing method in the above-mentioned embodiment.

[0201] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip executes the photo processing method in the above-mentioned method embodiments.

[0202] Among them, the electronic device, computer storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0203] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0205] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0207] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (RandomAccess Memory, RAM), disk or optical disk and other media that can store program code.

[0208] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. A photo processing method, applied to an electronic device, characterized in that: The method comprises: When the electronic device is in a preset state, acquiring a plurality of first photos that are not included in the photo collection; dividing the plurality of first photos into a plurality of photo sets; If the analysis of the photos in any of the plurality of photo sets is not completed, identifying the type of the photos that are not completed; If the analysis of all the unanalyzed photos is successful, a theme matching each photo is determined according to the type of each photo in the plurality of photo collections, and the photo is added to the photo collection corresponding to the theme.

2. The photo processing method according to claim 1, characterized in that: The method further comprises: If the analysis of all the photos in the photo collection is completed, the theme matching each photo is determined according to the type of each photo in the plurality of photo collections, and the photo is added to the photo collection corresponding to the theme.

3. The photo processing method according to claim 1, characterized in that: The method further comprises: If the analysis of any of the unanalyzed photos fails, it is determined that the generation of the photo collection corresponding to the multiple first photos fails.

4. The photo processing method according to claim 1, characterized in that: The method further comprises: If the electronic device is in the preset state again, a photo set including photos that failed to be analyzed is obtained, each photo that failed to be analyzed in the obtained photo set is analyzed, and the type of each photo that failed to be analyzed is identified.

5. The photo processing method according to claim 1, characterized in that: The preset state includes that the electronic device is charging with the screen turned off, the power of the electronic device is greater than or equal to the preset power, and the temperature of the electronic device is less than or equal to the preset temperature.

6. The photo processing method according to claim 1, characterized in that: When the electronic device is in a preset state, acquiring a plurality of first photos that are not included in the photo collection includes: Communicate with the power management service, display management service and temperature sensor of the electronic device through the status listener to monitor the screen on / off status, charging status, power and temperature of the electronic device; If the state listener determines that the electronic device is in the preset state, it generates a generation instruction for a photo collection, and in response to the generation instruction for the photo collection, obtains multiple first photos that are not included in the photo collection from the gallery database of the electronic device.

7. The photo processing method according to claim 6, characterized in that: When the electronic device is in a preset state, acquiring a plurality of first photos that are not included in the photo collection includes: When the electronic device is in the preset state, determining whether a gallery application of the electronic device includes a photo collection; If the gallery application does not include a photo collection, obtaining all photos in the gallery database; If the gallery application includes a photo collection, obtain photos in the gallery database that are not stored in a folder corresponding to the photo collection.

8. The photo processing method according to claim 6, characterized in that: When the electronic device is in a preset state, acquiring a plurality of first photos that are not included in the photo collection includes: Determining whether the photo information of each photo in the photo library database includes photo collection information; If the photo information of the photo includes the photo collection information, determining that the photo is included in the photo collection; If the photo information of the photo does not include the photo collection information, it is determined that the photo is not included in the photo collection.

9. The photo processing method according to claim 1, characterized in that: The step of dividing the plurality of first photos into a plurality of photo sets comprises: The obtained multiple first photos are clustered to obtain multiple second photos corresponding to multiple cluster categories respectively.

10. The photo processing method according to claim 9, characterized in that: The step of clustering the plurality of first photos to obtain a plurality of second photos corresponding to a plurality of cluster categories includes: The multiple first photos are time-clustered to obtain multiple second photos corresponding to multiple preset time ranges.

11. The photo processing method according to claim 9, characterized in that: The step of clustering the plurality of first photos to obtain a plurality of second photos corresponding to a plurality of cluster categories includes: The plurality of first photos are clustered by location to obtain a plurality of second photos corresponding to a plurality of location ranges.

12. The photo processing method according to claim 1, characterized in that: The types of photos that have not been analyzed include: Preprocessing the unanalyzed photos and inputting them into the target detection model; Extracting a feature map of the photo that has not been analyzed by the target detection model; According to the feature graph of the photo that has not been analyzed, extracting the region of interest of the photo that has not been analyzed by using the object detection model; According to the region of interest of the photo that has not been analyzed, extracting a feature map of the region of interest from the feature map of the photo that has not been analyzed by the target detection model; According to the features of the interest region of the photo that has not been analyzed, the interest region is classified by the target detection model to determine the type of the target object in the photo that has not been analyzed.

13. An electronic device, characterized in that: The electronic device comprises a memory and a processor: Wherein, the memory is used to store program instructions; The processor is used to read and execute the program instructions stored in the memory. When the program instructions are executed by the processor, the electronic device executes the photo processing method as described in any one of claims 1 to 12.

14. A chip coupled to a memory in an electronic device, characterized in that: The chip is used to control the electronic device to execute the photo processing method according to any one of claims 1 to 12.

15. A computer storage medium, characterized in that: The computer storage medium stores program instructions, and when the program instructions are executed on an electronic device, a processor of the electronic device executes the photo processing method according to any one of claims 1 to 12.

Images