Media data search method and electronic device

Abstract

The present disclosure provides a media data search method and electronic device, which are applied to the field of terminal technology. The method can help users directly locate the required media data. The method includes: obtaining media data; a first application classifies and processes the media data to obtain classification information corresponding to the media data, and updates the classification information corresponding to the media data to a gallery database; when a second application determines that the gallery database is updated, the second application obtains the classification information corresponding to the media data from the gallery database; a third application receives a user's selection operation of a first option among multiple classification options, and sends a media data acquisition request to the second application; the second application receives the media data acquisition request, generates a thumbnail of the media data corresponding to the first option based on the classification information corresponding to the media data, and sends the thumbnail of the media data corresponding to the first option to the third application; the third application receives and displays the thumbnail of the media data corresponding to the first option.

Description

Technical Field

[0001] The present disclosure relates to the field of terminal technology, and in particular to a media data search method and an electronic device. Background Art

[0002] When users use third-party applications (such as WeChat applications, Meitu applications, etc.), media data sharing scenarios may be involved. In media data sharing scenarios, users can use third-party applications to search for required media data among all media data (such as pictures, videos, etc.) on electronic devices, and then share the found media data with other users.

[0003] However, since the electronic device has a lot of media data, it may take a lot of time for the user to search, which makes the simple process complicated. For example, the electronic device has a WeChat application running on the foreground, and the WeChat application displays Figure 1 In response to the user clicking the photo control 102 in the dialogue interface 101, the mobile phone displays the following Figure 1 The image interface 103 shown in (b) in FIG. 1 includes a large number of images. Since there are so many images, it is often difficult for users to directly locate the desired image when searching, which will cost users a lot of time. Summary of the invention

[0004] The embodiments of the present disclosure provide a media data search method and an electronic device, which can help a user directly locate required media data, thereby improving the search efficiency of media data and improving user satisfaction.

[0005] To achieve the above objectives, the embodiments of the present disclosure adopt the following technical solutions:

[0006] In a first aspect, the present disclosure provides a media data search method, which is applied to an electronic device, and the electronic device includes a first application, a second application and a third application, and the method includes: acquiring media data; the first application classifies the media data to obtain classification information corresponding to the media data, and updates the classification information corresponding to the media data to a gallery database; when the second application determines that the gallery database is updated, the second application acquires the classification information corresponding to the media data from the gallery database; the third application receives a user's selection operation for a first option among multiple classification options, and sends a media data acquisition request to the second application, the first option being any one of a portrait option, a location option and a transaction option; the media data acquisition request is used to request a thumbnail of the media data corresponding to the first option; the multiple classification options are generated based on the classification information of the media data; the second application receives the media data acquisition request, and in response to the media data acquisition request, generates a thumbnail of the media data corresponding to the first option based on the classification information corresponding to the media data, and sends the thumbnail of the media data corresponding to the first option to the third application; the third application receives and displays the thumbnail of the media data corresponding to the first option.

[0007] Based on the media data search method of the first aspect, it can be known that the present disclosure classifies the media data through the first application, can obtain the classification information corresponding to the media data, and update the classification information corresponding to the media data to the gallery database. Since the third application sets multiple classification options, when the user uses the third application, if the user needs to find media data, the first option corresponding to the required media data can be filtered out from the multiple classification options, which greatly reduces the search range of the media data. Since the multiple classification options are generated based on the classification information of the media data, the classification information of the media data is stored in the gallery database, the third application cannot directly access the gallery database, and the second application will update the classification information of the media data in the gallery database in a timely manner. Therefore, when the user selects the first option from the multiple classification options, the third application can obtain the thumbnail of the media data corresponding to the first option through the second application. In this way, the search efficiency of media data can be improved, thereby improving user satisfaction.

[0008] In combination with the first aspect, in another possible implementation, the first application classifies the media data to obtain classification information corresponding to the media data, including: the first application performs face detection on the media data to determine whether the photographed object in the media data includes a face; when the photographed object in the media data does not include a face, the first application performs marking classification processing and location classification processing on the media data to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes category information corresponding to the media data and location information corresponding to the media data to be processed.

[0009] Based on this solution, it can be seen that when the first application classifies the media data, it first determines whether the photographed object in the media data includes a face. If the face is not included, the media data is further identified according to the mark classification process and the location classification process to obtain an accurate classification of the media data. In this way, when the user makes a selection from multiple classification options, the media data corresponding to each classification option can be accurately provided to the user.

[0010] In combination with the first aspect, in another possible implementation method, after determining whether the photographed subject in the media data includes a face, the method also includes: in the case that the photographed subject in the media data includes a face, a first application performs facial feature extraction processing and face clustering processing on the media data to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes portrait information corresponding to the media data; wherein the portrait information corresponding to the media data includes portrait album identification information corresponding to the media data, and the portrait album identification information corresponding to the media data is used to characterize the portrait data of the photographed subject in the media data.

[0011] Based on this solution, when the first application determines that the photographed object in the media data includes a face, the media data is subjected to face feature extraction processing and face clustering processing to obtain the specific situation of the portrait included in the photographed object in the media data, and the portrait album identification information is set according to the specific situation of the portrait. The portrait album identification information describes the portrait data of the photographed object in the media data in detail. In this way, it is convenient to further classify the media data based on the portrait album identification information in the future, thereby improving the search efficiency of the media data.

[0012] In combination with the first aspect, in another possible implementation method, the first application classifies the media data and obtains classification information corresponding to the media data. The method also includes: the first application constructs index data based on the classification information corresponding to the media data, the index data including the correspondence between the media data and the classification information corresponding to the media data; the first application sends the index data to the search service.

[0013] Based on this solution, it can be seen that the first application constructs index data based on the classification information corresponding to the media data, and sends the index data to the search service, which is equivalent to providing a search function for the electronic device, thereby enriching the user experience.

[0014] In combination with the first aspect, in another possible implementation method, the method also includes: the second application receives a media data search request from a third application, the media data search request includes a search term, and the media data search request is used to request to obtain media data corresponding to the search term; in response to the media data search request, the second application obtains the media data corresponding to the search term based on the index data in the search service, and sends the media data corresponding to the search term to the third application; the third application receives and displays the media data corresponding to the search term.

[0015] Based on this solution, since the third application cannot directly access the index data in the search service, while the second application can access the search service, when the user uses the third application, the third application can also use the second application to provide the user with a search function, thereby enriching the user's experience.

[0016] In combination with the first aspect, in another possible implementation method, the first option includes a first sub-portrait option and a second sub-portrait option in the portrait option; the first sub-portrait option corresponds to the first portrait, and the second sub-portrait option corresponds to the second portrait; in response to a media data acquisition request, the second application generates media data corresponding to the first option based on the classification information corresponding to the media data, including: in response to the media data acquisition request, the second application determines the first album tag information corresponding to the first portrait based on the portrait album identification information corresponding to the media data; the first album tag information corresponds to at least one first media data; the photographed object in the first media data includes the first portrait; determine whether the portrait album identification information corresponding to each first media data in at least one first media data contains the second album tag information, and determine the media data containing the second album tag information in at least one first media data as the second media data; the photographed object in the second media data includes at least the first portrait and the second portrait; the second application generates a thumbnail of the media data corresponding to the first option based on the second media data.

[0017] Based on this solution, it can be seen that the portrait album identification information can describe the portrait data of the subject in the media data. If the subject in the media data includes one portrait, the media data has only one portrait album identification information. If the subject in the media data includes multiple portraits, the media data has multiple portrait album identification information. Therefore, when the user selects multiple sub-portrait options (i.e., the first sub-portrait option and the second sub-portrait option), the media data corresponding to the multiple sub-portrait options can be obtained based on the portrait album identification information, thereby providing the user with media data containing multiple portraits (i.e., group photo images).

[0018] In combination with the first aspect, in another possible implementation, after the classification information corresponding to the media data is updated to the gallery database, the method further includes: the gallery database generates a transaction database according to the category information corresponding to the media data; generates a location database according to the location information corresponding to the media data; and generates a portrait database according to the portrait information corresponding to the media data; wherein the transaction database is used to characterize the correspondence between the media data and the category information corresponding to the media data; the location database is used to characterize the correspondence between the media data and the location information corresponding to the media data; and the portrait database is used to characterize the correspondence between the media data and the portrait information corresponding to the media data. Based on this solution, a functional introduction of a gallery database is provided.

[0019] In combination with the first aspect, in another possible implementation method, sending a thumbnail corresponding to media data to a third application includes: determining whether the size of the thumbnail of the media data corresponding to the first option exceeds a preset threshold; if the size of the thumbnail of the media data corresponding to the first option exceeds the preset threshold, scaling the thumbnail of the media data corresponding to the first option to obtain a scaled thumbnail of the media data corresponding to the first option; if the size of the thumbnail of the media data corresponding to the first option after scaling does not exceed the preset threshold, sending the scaled thumbnail of the media data corresponding to the first option to the third application.

[0020] Based on this scheme, it can be seen that since the size of data sent between two processes is strictly limited, before sending the thumbnail of the media data corresponding to the first option, it is also necessary to ensure that the thumbnail of the media data corresponding to the first option does not exceed the limit, so that the data interaction between the processes can proceed smoothly and the operating stability of the electronic device can be improved.

[0021] In combination with the first aspect, in another possible implementation, the preset threshold is 500 KB. An example of a preset threshold is provided.

[0022] In a second aspect, an embodiment of the present disclosure provides a media data search device, which can be applied to an electronic device to implement the method in the first aspect. The functions of the media data search device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions, such as an acquisition module, a processing module, a sending module, a receiving module, and a display module.

[0023] Among them, the acquisition module is configured to acquire media data, and when the second application determines that the gallery database is updated, the second application acquires the classification information corresponding to the media data from the gallery database. The processing module is configured to classify the media data for the first application, obtain the classification information corresponding to the media data, and update the classification information corresponding to the media data to the gallery database. The sending module is configured to receive the user's selection operation on the first option among multiple classification options as a third application, and send a media data acquisition request to the second application, the first option is any one of the portrait option, the location option and the transaction option; the media data acquisition request is used to request to obtain a thumbnail of the media data corresponding to the first option; the multiple classification options are generated based on the classification information of the media data; and send the thumbnail of the media data corresponding to the first option to the third application. The receiving module is configured to receive the media data acquisition request as the second application, and generate a thumbnail of the media data corresponding to the first option based on the classification information corresponding to the media data in response to the media data acquisition request. The display module is configured to receive and display the thumbnail of the media data corresponding to the first option as the third application.

[0024] In combination with the second aspect, in a possible implementation, the processing module is further configured as a first application to perform face detection processing on the media data to determine whether the photographed object in the media data includes a face; when the photographed object in the media data does not include a face, the first application performs marking classification processing and location classification processing on the media data to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes category information corresponding to the media data and location information corresponding to the media data to be processed.

[0025] In combination with the second aspect, in a possible implementation method, the processing module is also configured to, when the photographed subject in the media data includes a face, perform facial feature extraction processing and face clustering processing on the media data by the first application to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes portrait information corresponding to the media data; wherein the portrait information corresponding to the media data includes portrait album identification information corresponding to the media data, and the portrait album identification information corresponding to the media data is used to characterize the portrait data of the photographed subject in the media data.

[0026] In conjunction with the second aspect, in a possible implementation, the processing module is further configured so that the first application constructs index data based on the classification information corresponding to the media data, and the index data includes a correspondence between the media data and the classification information corresponding to the media data. The sending module is further configured so that the first application sends the index data to the search service.

[0027] In combination with the second aspect, in a possible implementation, the receiving module is also configured so that the second application receives a media data search request from a third application, the media data search request includes a search term, and the media data search request is used to request to obtain media data corresponding to the search term; the processing module is also configured so that in response to the media data search request, the second application obtains the media data corresponding to the search term based on the index data in the search service, and sends the media data corresponding to the search term to the third application; the display module is also configured so that the third application receives and displays the media data corresponding to the search term.

[0028] In combination with the second aspect, in a possible implementation method, the first option includes a first sub-portrait option and a second sub-portrait option in the portrait option; the first sub-portrait option corresponds to the first portrait, and the second sub-portrait option corresponds to the second portrait; the processing module is also configured to respond to the media data acquisition request, and the second application determines the first album marking information corresponding to the first portrait based on the portrait album identification information corresponding to the media data; the first album marking information corresponds to at least one first media data; the photographed object in the first media data includes the first portrait; determine whether the portrait album identification information corresponding to each first media data in at least one first media data contains the second album marking information, and determine the media data containing the second album marking information in at least one first media data as the second media data; the photographed object in the second media data includes at least the first portrait and the second portrait; the second application generates a thumbnail of the media data corresponding to the first option based on the second media data.

[0029] In combination with the second aspect, in a possible implementation method, the processing module is also configured to generate a transaction database from the gallery database according to the category information corresponding to the media data; generate a location database according to the location information corresponding to the media data; and generate a portrait database according to the portrait information corresponding to the media data; wherein the transaction database is used to characterize the correspondence between the media data and the category information corresponding to the media data; the location database is used to characterize the correspondence between the media data and the location information corresponding to the media data; and the portrait database is used to characterize the correspondence between the media data and the portrait information corresponding to the media data.

[0030] In combination with the second aspect, in a possible implementation method, the processing module is also configured to determine whether the size of the thumbnail of the media data corresponding to the first option exceeds a preset threshold; if the size of the thumbnail of the media data corresponding to the first option exceeds the preset threshold, scale the thumbnail of the media data corresponding to the first option to obtain a scaled thumbnail of the media data corresponding to the first option; if the size of the thumbnail of the media data corresponding to the first option after scaling does not exceed the preset threshold, send the scaled thumbnail of the media data corresponding to the first option to a third application.

[0031] In conjunction with the second aspect, in a possible implementation, the preset threshold is 500 KB.

[0032] In a third aspect, the present disclosure provides an electronic device, comprising: a memory, a display screen, and one or more processors; the memory, the display screen, and the processor are coupled. The memory is used to store computer program codes, and the computer program codes include computer instructions; when the electronic device is running, the processor is used to execute one or more computer instructions stored in the memory, so that the electronic device executes the data processing method as described in any one of the first aspects above.

[0033] In a fourth aspect, the present disclosure provides a computer storage medium, comprising computer instructions, which, when executed on an electronic device, enables the electronic device to execute any of the data processing methods in the first aspect.

[0034] In a fifth aspect, the present disclosure provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the data processing method as described in any one of the first aspects.

[0035] In a sixth aspect, a device (for example, the device may be a chip system) is provided, the device including a processor for supporting a first terminal device to implement the functions involved in the first aspect. In one possible design, the device also includes a memory for storing necessary program instructions and data for the first terminal device. When the device is a chip system, it may be composed of a chip, or may include a chip and other discrete devices.

[0036] It should be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is one of the display schematic diagrams provided in the embodiment of the present disclosure.

[0038] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure.

[0039] Figure 3 This is one of the software structure diagrams of an electronic device provided in an embodiment of the present disclosure.

[0040] Figure 4 The second schematic diagram of the software structure of an electronic device provided in an embodiment of the present disclosure.

[0041] Figure 5 This is one of the flowcharts of a media data search method provided in an embodiment of the present disclosure.

[0042] Figure 6 The present invention provides a second display schematic diagram of an embodiment of the present invention.

[0043] Figure 7 The present invention provides a third display schematic diagram of an embodiment of the present invention.

[0044] Figure 8 This is a fourth display schematic diagram provided for an embodiment of the present disclosure.

[0045] Fig. 9 This is a fifth display schematic diagram provided for an embodiment of the present disclosure.

[0046] Fig.10 This is a sixth display schematic diagram provided for an embodiment of the present disclosure.

[0047] Fig.11 This is a seventh display schematic diagram provided for an embodiment of the present disclosure.

[0048] Fig.12 The second flowchart of a media data search method provided in an embodiment of the present disclosure.

[0049] Fig.13 A schematic diagram of the structure of a media data search device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] The technical solution in the embodiment of the present disclosure will be described below in conjunction with the drawings in the embodiment of the present disclosure. Among them, in the description of the present disclosure, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present disclosure is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. And, in the description of the present disclosure, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present disclosure, in the embodiments of the present disclosure, the words "first", "second" and the like are used to distinguish between the same items or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like do not necessarily limit the differences. At the same time, in the embodiments of the present disclosure, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present disclosure should not be interpreted as being more preferred or more advantageous than other embodiments or design. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for easy understanding.

[0051] In addition, the network architecture and business scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Ordinary technicians in this field can know that with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0052] The implementation of this embodiment will be described in detail below with reference to the accompanying drawings.

[0053] Exemplarily, a media data search method provided in an embodiment of the present application can be applied to electronic devices such as mobile phones, tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, cellular phones, personal digital assistants (PDAs), personal computers (PCs), augmented reality (AR) and virtual reality (VR) devices, wearable electronic devices, and virtual reality devices. The embodiments of the present disclosure do not impose any special restrictions on the specific form of the electronic device. For example, the electronic device can be a mobile phone.

[0054] For example, Figure 2 A schematic diagram of the structure of an electronic device is shown. Figure 2 As shown, the electronic device may include a processor 210, an external memory interface 230, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a button 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc. Among them, the sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, an air pressure sensor 280C, a magnetic sensor 280D, an acceleration sensor 280E, a distance sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, etc.

[0055] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0056] The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0057] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0058] The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 210, and thus improves the efficiency of the system.

[0059] In some embodiments, the processor 210 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0060] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor and baseband processor.

[0061] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0062] The mobile communication module 250 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to electronic devices. The mobile communication module 250 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 250 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 250 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 250 can be set in the same device as at least some of the modules of the processor 210.

[0063] The wireless communication module 260 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to electronic devices. The wireless communication module 260 can be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 210. The wireless communication module 260 can also receive the signal to be sent from the processor 210, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0064] In some embodiments, the antenna 1 of the electronic device is coupled to the mobile communication module 250, and the antenna 2 is coupled to the wireless communication module 260, so that the electronic device can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc.

[0065] The electronic device implements the display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs that execute program instructions to generate or change display information.

[0066] The display screen 294 is used to display images, videos, etc. The display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 294, where N is a positive integer greater than 1.

[0067] The electronic device can realize the shooting function through the ISP, the camera 293, the video codec, the GPU, the display screen 294 and the application processor, etc. In some embodiments, the electronic device can include 1 or N cameras 293, where N is a positive integer greater than 1.

[0068] The internal memory 221 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 210 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 221. The internal memory 221 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data established during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0069] In addition, an operating system runs on top of the above components, such as Etc. Application programs can be installed and run on the operating system. In other embodiments, there can be multiple operating systems running in the electronic device.

[0070] It should be understood that Figure 2 The hardware modules included in the electronic device shown are only described for example, and do not limit the specific structure of the electronic device. In fact, the electronic device provided by the embodiment of the present disclosure may also include other hardware modules that have an interactive relationship with the hardware modules illustrated in the figure, which are not specifically limited here. For example, the electronic device may also include a flashlight, a micro-projection device, etc. For another example, if the electronic device is a PC, then the electronic device may also include components such as a keyboard and a mouse.

[0071] It is understandable that, generally speaking, the realization of electronic device functions requires not only hardware support but also software cooperation.

[0072] The software system of the electronic device may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. Taking the system as an example, the software structure of the electronic device is illustrated.

[0073] Figure 3 It is a software structure block diagram of the electronic device provided in the embodiment of the present application.

[0074] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and the system library (also called the Native layer), and the kernel layer.

[0075] The application layer can include a series of application packages. Figure 3 As shown, the application package may include applications such as camera, gallery, calendar, phone, map, navigation, WLAN, Bluetooth, music, video, short message, etc. The application framework layer provides an application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0076] It is understandable that the application layer may include multiple applications, and the types of the multiple applications may be the same or different. For example, the application layer may include multiple shopping applications, and the multiple shopping applications are of the same type, all of which are shopping applications. The application layer may also include a map application and a communication application, and the map application and the communication application are of different types.

[0077] like Figure 3 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0078] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0079] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0080] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0081] The phone manager is used to provide communication functions for electronic devices, such as the management of call status (including answering, hanging up, etc.).

[0082] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0083] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0084] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0085] The core library consists of two parts: one is the function that the Java language needs to call, and the other is The core library.

[0086] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0087] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0088] The surface manager is used to manage the display subsystem and provide the fusion of two-dimensional and three-dimensional layers for multiple applications.

[0089] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats.

[0090] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0091] A 2D graphics engine is a drawing engine for 2D drawings.

[0092] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0093] The methods in the following embodiments can all be implemented in an electronic device having the above hardware structure or software structure.

[0094] In some examples, in order to implement the multimedia resource display method provided in the embodiments of the present application, Figure 3 Based on the software structure diagram of the electronic device shown in Figure 4 As shown, in addition to the gallery application (application, APP), the application layer can also include pending applications (i.e., pending APPs), media data middle-end applications (i.e., media data middle-end APPs), media databases, and gallery databases, etc.

[0095] The APP to be processed is an APP that cannot directly call the classification results of media data from the gallery APP. For example, the APP to be processed can be a third-party APP (for example, Meituan APP, WeChat APP, etc.) or a system APP. Media data includes pictures, videos, files, etc.

[0096] The APP to be processed includes a Media Data Kit (MediaDate Kit). The APP to be processed can obtain the corresponding media data from the media data middle platform APP through the MediaDate Kit.

[0097] The media data middle platform APP includes multiple functional modules, such as kit authentication module, metadata access module, thumbnail access module, media search module, permission check module, media metadata database, thumbnail management module, search toolkit (Search Kit) and metadata synchronization module, etc.

[0098] The kit authentication module is used to detect whether the APP to be processed has been successfully registered on the developer website (for example, Honor Developer Platform). If the APP to be processed is successfully registered on the developer website, it means that the APP to be processed has the permission to access other applications on the developer website.

[0099] Specifically, when the media data middle station APP receives a media data acquisition request from the APP to be processed, the media data middle station APP responds to the media data acquisition request and sends an authentication request to the kit authentication module. The authentication request is used to confirm whether the APP to be processed is successfully registered on the developer website. The authentication request includes the package name, service name and other information of the APP to be processed. After receiving the authentication request, the kit authentication module sends the authentication request to the authentication middle station. The authentication middle station calls the authentication service to authenticate the APP to be processed to obtain the authentication result. Afterwards, the media data middle station APP can determine whether to respond to the media data acquisition request based on the authentication result and search for the media data corresponding to the data acquisition request.

[0100] After determining that the APP to be processed has the authority to access other applications in the developer's website, the media data middle platform APP can access the metadata access module, the thumbnail access module or the media search module according to the media data acquisition request.

[0101] If the media data acquisition request is used to request access to metadata, the media data middle station calls the metadata access module to obtain the metadata corresponding to the media data access request. In other words, the metadata access module is used to respond to the media data access request triggered by the user in the pending APP and output the metadata corresponding to the media data.

[0102] If the media data acquisition request is used to request access to a thumbnail, the media data middle station calls the thumbnail access module to obtain the thumbnail corresponding to the media data access request. In other words, the thumbnail access module is used to respond to the media data access request triggered by the user in the pending APP and output the thumbnail corresponding to the media data.

[0103] If the media data acquisition request is used to request access to search results, the media data middle station calls the media search module to obtain the search results corresponding to the media data access request. In other words, the media search module is used to respond to the media data access request triggered by the user in the pending APP and output the search results corresponding to the media data.

[0104] The permission checking module is used to determine whether the APP to be processed has the permission to read and / or store the media data, wherein the permission to read and / or store can be a permission granted by the user.

[0105] The media metadata database is used to store data in the media database and the gallery database.

[0106] The thumbnail management module is used to generate thumbnails of each picture corresponding to the first portrait, thumbnails of each picture corresponding to the second portrait, and at least one thumbnail corresponding to the search result in response to the first portrait thumbnail acquisition request, the second portrait thumbnail acquisition request, and the search thumbnail acquisition request sent by the pending APP, and send thumbnails of each picture corresponding to the first portrait, thumbnails of each picture corresponding to the second portrait, and at least one thumbnail corresponding to the search result to the pending APP. The first portrait thumbnail acquisition request is used to request to obtain at least one thumbnail corresponding to the first portrait. The second portrait thumbnail acquisition request is used to request to obtain at least one thumbnail corresponding to the second portrait. The search thumbnail acquisition request is used to request to obtain at least one thumbnail corresponding to the search result.

[0107] The search management module includes a search kit (Search Kit). The search management module is used to respond to the search query request sent by the pending APP, send the search request to the search service (SearchService); receive the search results corresponding to the search information fed back from the search service; and send the search results corresponding to the search information to the pending APP. Among them, the search query request is used to request to obtain the search results corresponding to the search information. The search request includes search information. The search information includes search terms.

[0108] The metadata management module is used to monitor whether data in the media database and the gallery database is updated. When data in the media database and the gallery database is updated, the metadata management module synchronizes the updated data in the media database and the gallery database, and stores the updated data in the media metadata database.

[0109] The media database is used to store all media data in the electronic device. The media data can be at least one of files, music, pictures, videos, texts, downloaded data, etc.

[0110] The gallery database is used to store media data corresponding to the gallery APP. The media data corresponding to the gallery APP may include at least one of pictures, videos, etc.

[0111] The Gallery APP includes multiple functional modules, such as albums, highlights, creations, portrait modules, classification modules, location processing modules and search modules.

[0112] Among them, Albums, Wonderful Moments, and Creations are sub-functions in the Gallery APP, which can enrich the user experience of the Gallery APP.

[0113] The portrait module can be used to perform face detection, face feature extraction and face clustering on all media data in the media database to obtain portrait information; and save the portrait information to the gallery database.

[0114] The classification module can be used to mark and classify the media data in the media database that does not include faces, obtain category information, and save the category information to the gallery database.

[0115] The location processing module can be used to perform location classification processing on the category information to obtain location information; and save the location information to the gallery database.

[0116] The search module includes a search kit. The search module can respond to an image search request triggered by a user in the gallery app by sending the image search request to the search service, receiving search results corresponding to the image search information fed back from the search service, and displaying the search results corresponding to the image search information in the gallery app. The image search request is used to request to obtain search results corresponding to the image search information. The image search request is used to request to search for media data related to the search term entered by the user, and the image search request includes the search term entered by the user.

[0117] The thumbnail database is used to store all thumbnail data in the electronic device. Exemplarily, the thumbnail database can be a key-value database. The key-value database is a database that stores data in correspondence with a key value, and each key value corresponds to a unique value, which includes the stored data.

[0118] The fields corresponding to each thumbnail stored in the key-value database include a key field and a value field. The key field is used to identify different thumbnails. Thumbnails with different key field values ​​correspond to different original images, and thumbnails with the same key field value correspond to the same original image. The electronic device can determine whether the corresponding thumbnail has been stored in the thumbnail database based on the key field value corresponding to the thumbnail.

[0119] Below Figure 4 Based on the software structure diagram of the electronic device shown, combined with Figure 5 The media data search method provided in the embodiment of the present application is schematically described. The method can be applied to the above-mentioned electronic device, taking the electronic device as a mobile phone as an example, Figure 5 As shown, the multimedia resource display method may include the following steps 501 to 547.

[0120] Step 501: When the electronic device is in the target state, the portrait module in the gallery APP performs face detection on multiple images to be detected to obtain a face detection result for each of the multiple images to be detected.

[0121] The multiple images to be detected are obtained by the portrait module from the media database. The face detection result of each image to be detected includes whether the subject in the image to be detected includes a face and whether the subject in the image to be detected does not include a face.

[0122] according to Figure 4It can be seen that the media database stores all media data in the electronic device. In some examples, when the media data in the media database is updated, the media database can send the updated media data to the portrait module. The updated media data includes multiple images to be detected.

[0123] In other examples, the portrait module may monitor in real time whether the media data in the media database is updated. When it is detected that the media data in the media database is updated, the portrait module may obtain the updated media data from the media database.

[0124] In some exemplary embodiments, the target state is that the electronic device is in a charging state and / or a screen-off state.

[0125] When the electronic device is in a charging state and / or a screen-off state, the electronic device can pull up the portrait module in the gallery APP to enable the portrait module to perform face detection on multiple images to be detected. After the portrait module is pulled up, the portrait module performs face detection on multiple images to be detected obtained from the media database to identify whether the multiple images to be detected include faces.

[0126] When the electronic device is in a charging state, if the portrait module performs face detection on multiple images to be detected, the external power supply can be used to provide power to the electronic device, avoiding the use of the electronic device's own battery power, thereby extending the battery life. In addition, in the charging state, the electronic device can use the external power supply to provide a more stable power supply, thereby improving the performance and response speed of the electronic device.

[0127] When an electronic device is in the screen-off state, the user generally does not use the electronic device, so the screen of the electronic device rarely lights up, thereby saving power. In addition, the electronic device requires fewer resources when it is in the charging state and / or the screen-off state, and there is no resource competition when the portrait module is used for face detection, thereby improving the processing efficiency of face detection.

[0128] In some other exemplary embodiments, the target state is that the electronic device is in a charging state and / or a screen-off state, and the power level of the electronic device is greater than a preset threshold, for example, the preset threshold is 50%.

[0129] When the electronic device is in charging state and / or screen-off state, and the battery level of the electronic device is greater than 50%, if the portrait module performs face detection on multiple images to be detected, task interruption or device shutdown due to insufficient battery can be avoided.

[0130] In some examples, the portrait module performs face detection on multiple images to be detected, which may include the following steps: First, the portrait module obtains multiple images to be detected from a media database. Then, when the electronic device is in a target state, the portrait module preprocesses the multiple images to be detected. Among them, preprocessing the multiple images to be detected includes adjusting the size of the images to be detected, normalizing the images to be detected, and performing noise reduction on the images to be detected. Then, using an image feature extraction algorithm, facial feature information is extracted from the multiple images to be detected, and the facial feature information is used to characterize some local patterns, textures, etc. of the face. For example, the image feature extraction algorithm may be Haar features (Haar-like features, Haar), directional gradient histogram features (histogram of oriented gradients, HOG), etc. Then, using a machine learning algorithm such as a classifier or a regressor, the extracted features are compared with known face or non-face samples to determine whether there is a face in the image to be detected. Commonly used algorithms may be cascade classifiers and support vector machines (SVM), etc. For example, the cascade classifier can be a Viola-Jones object detection framework (Viola-Jones algorithm). If a face is detected, the position and size of the face can be marked. For example, the position and size of the face can be marked by a rectangular frame or an elliptical frame. Finally, a post-processing operation is performed on the detected face area to remove redundant overlapping marking frames. For example, the post-processing operation can be a non-maximum suppression (NMS) process. It is understandable that the process of performing face detection on multiple images to be detected by the portrait module can also be implemented according to other algorithms and models, which is subject to actual needs and is not limited in this application.

[0131] After the portrait module performs face detection on multiple images to be detected, it can determine whether each of the multiple images to be detected includes a face. If a face is included, the image to be detected including the face can be further processed. If a face is not included, the image to be detected not including the face is divided into other dimensions to facilitate more detailed classification of the image to be detected later.

[0132] Step 502: When the face detection result of the first image to be detected is that the photographed object in the first image to be detected does not include a face, the portrait module sends a classification request to the classification module in the gallery APP.

[0133] The plurality of images to be detected include a first image to be detected and a second image to be detected. The subject in the first image to be detected does not include a face. The subject in the second image to be detected includes a face.

[0134] The classification request is used to request to perform label classification processing and location classification processing on the first image to be detected, so as to obtain the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected. The classification request includes the first image to be detected.

[0135] When the portrait module performs face detection on the first image to be detected and determines that the photographed object in the first image to be detected does not include a face, it is necessary to further clarify the category to which the first image to be detected belongs. In some examples, the classification module can be used to further clarify the category to which the first image to be detected belongs.

[0136] Step 503: The classification module receives the classification request.

[0137] Step 504: In response to the classification request, the classification module performs labeling and classification processing on the first image to be detected to obtain category information corresponding to the first image to be detected.

[0138] The category information corresponding to the first image to be detected includes an image identifier of the first image to be detected, a classification label corresponding to the first image to be detected, and a classification probability corresponding to the classification label. The classification probability is used to indicate the probability that the first image to be detected belongs to the classification label. Exemplarily, the classification probability may be a confidence level.

[0139] After the classification module receives the classification request, the classification module may perform a marking classification process on the first image to be detected in response to the classification request to obtain the category information corresponding to the first image to be detected, so as to display the first image to be detected under a suitable classification set according to the category information corresponding to the first image to be detected.

[0140] In some examples, the classification label includes a first label and a second label. The first label can also be called a large classification label, and the second classification can also be called a small classification label. The first label and the second label are divided based on common scenarios. The division result of the first label is relatively rough. The division result of the second label is more refined.

[0141] Exemplarily, the first label can be represented by category_id. The second label can be represented by Sub_label. The specific contents of category_id and Sub_label can be numbers. One number represents one category. For example, the category_id corresponding to the image to be detected 1 is 64. 64 is used to represent the landscape category. In other words, the category label of the image to be detected 1 is landscape.

[0142] It can be understood that the categories corresponding to different numbers can be preset, and the present disclosure does not limit the specific category corresponding to each number.

[0143] For example, the first tag may include 5 sub-tags, namely food, scenery, transportation, sports and documents. The second tag may include 22 sub-tags, namely birthday cake, mountain, flower, city view, ocean, night view, building, waterfall, rainbow, fountain, fireworks, business card, PPT, ID card, passport, bank card, graduation photo, dinner, cat, panda, airplane and watch.

[0144] It is understandable that the contents in the first tag and the second tag may be increased or deleted according to actual conditions, and the present disclosure does not limit this.

[0145] After the classification module performs labeling and classification processing on the first image to be detected, a classification label corresponding to the first image to be detected can be obtained. The classification label corresponding to the first image to be detected can include only the first label, or can include both the first label and the second label.

[0146] When the classification label of the first image to be detected includes both the first label and the second label, the first image to be detected may correspond to one first label and one or more second labels. For example, the photographed objects in the image to be detected 1 include a rainbow and a fountain, so the image to be detected 1 corresponds to one first label and two second labels, the first label is landscape, and the second label is rainbow and fountain.

[0147] In some examples, the classification module performs a labeling classification process on the first image to be detected, and obtaining the category information corresponding to the first image to be detected may include the following steps: first, obtain the first image to be detected from the classification request. Secondly, preprocess the first image to be detected to improve the accuracy and robustness of the classification. Among them, the preprocessing may include operations such as resizing, normalization, denoising, and contrast enhancement. Thirdly, extract features from the first image to be detected using a feature extraction algorithm. Among them, the extracted features may be low-level features extracted directly from pixel values, or high-level features learned through deep learning models such as convolutional neural networks (CNN). Then, the extracted features are converted into feature representations that can be processed by the classifier. For example, the feature representation may be obtained by vectorization, dimensionality reduction, image pyramid, etc. Then, using the labeled training data, the feature representation is used as input data to train the initial classifier model. For example, the initial classifier model may be SVM, decision tree, random forest, neural network, etc. Finally, the feature representation of the first image to be predicted is input into the trained classifier model for classification prediction to input the category information corresponding to the first image to be detected. The category information may include a classification label corresponding to the first image to be detected and a classification probability corresponding to the classification label.

[0148] Step 505: The classification module performs location classification processing on the first image to be detected, obtains location information corresponding to the first image to be detected, and sends the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected to the gallery database.

[0149] The location information corresponding to the first image to be detected is used to represent the shooting location of the first image to be detected. Exemplarily, the location category information of the first image to be detected can be represented by geo_name.

[0150] The classification module performs location classification processing on the first image to be detected, and the process of obtaining the location information corresponding to the first image to be detected can be: the classification module queries the basic information of the first image to be detected. The basic information of the first image to be detected includes the geographical coordinates corresponding to the shooting location of the first image to be detected. Based on the geographical coordinates corresponding to the shooting location of the first image to be detected, the location information of the first image to be detected is obtained. It can also be a pre-trained location classification model, the first image to be detected is input into the trained location classification model, and the location classification model outputs the location information corresponding to the first image to be detected. For example, the location information corresponding to the first image to be detected is Beijing. It can be understood that the present disclosure does not limit the process of determining the location information corresponding to the first image to be detected, as long as it meets actual needs.

[0151] In some examples, the location information corresponding to the first image to be detected may include the image longitude of the first image to be detected, the image latitude of the first image to be detected, the location category corresponding to the first image to be detected, and the location category corresponding to the first image to be detected. Among them, the location category may include the area (e.g., Beijing). The location category may include a region (e.g., Haidian), avenues (e.g., Buy and Sell Street), secondary roads (e.g., No. 40), and location feature names (e.g., Xiangshan Park), etc.

[0152] After obtaining the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected, the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected can be sent to the gallery database, so that the subsequent media data middle platform APP can obtain the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected through the gallery database.

[0153] Step 506: The gallery database receives the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected, and generates a transaction database and a location database based on the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected.

[0154] After the gallery database receives the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected, the gallery database can generate a transaction database and a location database based on the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected. The transaction database and the location database are then synchronized to the media data middle station APP, so that when the media data middle station APP receives the media data query request of the APP to be processed, it can quickly feedback the query results of the media data to the APP to be processed based on the transaction database and the location database.

[0155] In some embodiments, generating a transaction database and a location database based on the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected can be to generate a transaction database based on the category information corresponding to the first image to be detected and the first image to be detected. Generate a location database based on the location information corresponding to the first image to be detected and the first image to be detected. Wherein, the transaction database is used to characterize the corresponding relationship between the category information corresponding to the first image to be detected and the first image to be detected. The location database is used to characterize the corresponding relationship between the first image to be detected and the location information corresponding to the first image to be detected.

[0156] Exemplarily, the transaction database may describe the relationship between the first image to be detected and the category information corresponding to the first image to be detected in the form of a table. The location database may also describe the relationship between the first image to be detected and the location information corresponding to the first image to be detected in the form of a table. It is understandable that the transaction database and the location database may also be in other forms, and the present disclosure does not limit this.

[0157] For example, the transaction database includes a transaction information table and a transaction feature table. The transaction information table may be an image_classify_info table. The transaction feature table may be an image_classify_feature table.

[0158] The transaction information table is used to characterize the classification label corresponding to the first image to be detected in the electronic device and the classification probability of the classification label. The transaction information table may include multiple sub-transaction information data. Each sub-transaction information data includes at least a sequence number, an image ID of the image to be detected, an image identifier of the image to be detected (for example, each image to be detected can correspond to a unique identifier, which can be represented by a hash value (hash)), a first label, a classification probability corresponding to the first label, a second label, and a classification probability corresponding to the second label. Exemplarily, the classification probability corresponding to the first label can be represented by category_confidence. The classification probability corresponding to the second label can be represented by sub_label_confidence.

[0159] The transaction feature table is used to characterize the image features of the first image to be detected in the electronic device. The transaction feature table may include multiple sub-transaction feature data. Each sub-transaction feature data includes at least a sequence number, an image ID of the image to be detected, an image identifier (e.g., a hash value) of the image to be detected, an image feature of the image to be detected, and an image address of the image to be detected. The image feature of the image to be detected may be an image feature extracted by a feature extraction algorithm. The image feature may be represented by binary data.

[0160] The location database includes a location dictionary table and a location knowledge table. The location dictionary table may be a t_geo_dictionary table. The location knowledge table may be a t_geo_konwledge table.

[0161] The location dictionary table is used to represent all location classifications corresponding to multiple images to be detected in the electronic device. The location dictionary table may include multiple sub-location dictionary data. Each sub-location dictionary data includes at least a sequence number, a location ID, a language type corresponding to the location, and a location name. Among them, the language type corresponding to the location is related to the language type set by the electronic device.

[0162] The location knowledge base is used to characterize the location information corresponding to multiple images to be detected in the electronic device. The location knowledge base may include multiple sub-location knowledge data. Each sub-location knowledge data includes at least a sequence number, an image ID of the image to be detected, an image longitude of the image to be detected, an image latitude of the image to be detected, a major location classification of the image to be detected, and a minor location classification of the image to be detected. It is understandable that when displaying the image to be detected, usually only the major location classification of the image to be detected is displayed. The minor location classification is generally used in search scenarios.

[0163] At present, the present disclosure uses category information and location information as the image classification basis of the image to be detected to classify the first image to be detected. And generates a transaction database and a location database based on the classification result of the first image to be detected. It is understandable that with the development of technology and the refinement of classification requirements, the scheme of the present disclosure can also be referred to, and the first image to be detected can be classified more finely according to the new image classification basis to obtain more detailed classification results. In this way, more classification options can be provided to users, which is convenient for users to quickly locate media data.

[0164] Step 507: When the face detection result of the second image to be detected is that the photographed object in the second image to be detected includes a face, the portrait module performs face feature extraction and face clustering on the second image to be detected to obtain portrait information corresponding to the second image to be detected.

[0165] Among them, the portrait information corresponding to the second image to be detected includes portrait album identification information. The portrait album identification information is used to characterize the portrait data of the subject in the image to be detected. The portrait data of the subject in the image to be detected is used to indicate the number of portraits included in the shooting correspondence in the image to be detected, and the portrait album identification information is used to distinguish each portrait. The subject in an image to be detected can be one portrait or multiple portraits.

[0166] When the photographed object in an image to be detected includes a portrait, the image to be detected can be included in a portrait album set. A portrait album set corresponds to a portrait album identification information. Therefore, the image to be detected can correspond to a portrait album identification information.

[0167] When the photographed object in an image to be detected includes multiple portraits, the image to be detected can be classified into multiple portrait album sets. Therefore, the image to be detected can correspond to multiple portrait album identification information.

[0168] Exemplarily, the portrait album identification information may be represented by group_tag.

[0169] Afterwards, the electronic device can generate multiple portrait albums based on the portrait album identification information, and display the multiple portrait albums in the gallery APP. A portrait album includes at least one image to be detected, and the subjects in at least one image to be detected all include the same portrait.

[0170] Each portrait album set in the plurality of portrait album sets includes at least one image to be detected. Each image to be detected in a portrait album set includes the same portrait. The images to be detected in different portrait album sets may be repeated.

[0171] In some examples, the facial information of the second image to be detected includes portrait album identification information, the image address of each image to be detected in the second image to be detected, feature data of the portrait in each image to be detected in the second image to be detected, the image identification of each image to be detected in the second image to be detected, the position of each portrait in each image to be detected in the second image to be detected (for example, the x-coordinate, y-coordinate, length and width of the portrait corresponding to the portrait), and the portrait similarity of each portrait in each image to be detected in the second image to be detected. Among them, the portrait similarity is used to characterize the accuracy of the recognized portrait.

[0172] Exemplarily, the feature data of the portrait in each image to be detected in the second image to be detected can be represented by a features tag. The specific content of the features tag is generally binary data. The image identifier of each image to be detected in the second image to be detected can be represented by a hash value. The image identifier of each image to be detected in the second image to be detected can also be represented by other types of data, which is not limited by the present disclosure.

[0173] In some examples, when the photographed object in the second image to be detected includes a face, the portrait module can extract face features from the second image to be detected to obtain feature data of the portrait in each image to be detected in the second image to be detected. Based on the feature data of the portrait in each image to be detected in the second image to be detected, all portraits involved in each image to be detected in the second image to be detected can be identified.

[0174] For example, the second image to be detected includes image a to be detected, image b to be detected, image c to be detected, and image d to be detected. There are two portraits in image a to be detected, namely, a portrait of user A and a portrait of user B. There is one portrait in image b to be detected, namely, a portrait of user C. There are two portraits in image c to be detected, namely, a portrait of user A and a portrait of user C. There is one portrait in image d to be detected, namely, a portrait of user B.

[0175] After all the human figures involved in each image to be detected in the second image to be detected are obtained, face clustering can be performed on the second image to be detected based on all the human figures involved in each image to be detected in the second image to be detected to obtain a portrait album set.

[0176] In some examples, based on all the faces involved in each image to be detected in the second image to be detected, performing face clustering on the second image to be detected to obtain a portrait album set may include the following steps: based on all the portraits involved in each image to be detected in the second image to be detected, taking portrait 1 among all the portraits as the standard portrait of portrait album set 1, traversing each image to be detected in turn, performing face clustering on all images to be detected containing portrait 1, thereby obtaining a portrait album set 1 corresponding to portrait 1. Then, referring to this method, portrait album sets corresponding to other portraits in all the portraits are obtained in turn.

[0177] For example, after it is found that the image a to be detected includes the portrait of user A and the portrait of user B; the image b to be detected includes the portrait of user C; the image c to be detected includes the portrait of user A and the portrait of user C; and the image d to be detected includes the portrait of user B, the portrait of user A is first used as the standard portrait of the portrait album set A, and the images a to be detected, the images b to be detected, the images c to be detected, and the images d to be detected are traversed in sequence, and the images a to be detected and the images c to be detected are clustered, so as to obtain the portrait album set A corresponding to the portrait of user A. Then, referring to this method, the portrait album set B corresponding to the portrait of user B and the portrait album set C corresponding to the portrait of user C are obtained in sequence.

[0178] Through face clustering processing, it can be known that portrait album set A includes all images to be detected corresponding to the portraits of user A. That is, portrait album set A contains image a to be detected and image c to be detected. Portrait album set B includes all images to be detected corresponding to the portraits of user B. That is, portrait album set B contains image a to be detected and image d to be detected. Portrait album set C includes all images to be detected corresponding to the portraits of user C. That is, portrait album set C contains image b to be detected and image c to be detected. Portrait album set A, portrait album set B and portrait album set C all correspond to different portrait album identification information.

[0179] In addition, since a portrait album set corresponds to a portrait album identification information, for example, portrait album set A corresponds to group_tag1, portrait album set B corresponds to group_tag2, and portrait album set C corresponds to group_tag3.

[0180] For the image a to be detected, since the image a to be detected belongs to both the portrait album set A and the portrait album set B, the image a to be detected can correspond to two portrait album identification information, that is, the image a to be detected corresponds to group_tag1 and group_tag2.

[0181] The image b to be detected only belongs to the portrait album set C, so the image b to be detected corresponds to one portrait album identification information. That is, the image b to be detected corresponds to group_tag3.

[0182] The image c to be detected belongs to both portrait album set A and portrait album set C, so the image c to be detected can correspond to two portrait album identification information, that is, the image c to be detected corresponds to group_tag1 and group_tag3.

[0183] The image d to be detected only belongs to the portrait album set B, so the image b to be detected corresponds to one portrait album identification information. That is, the image b to be detected corresponds to group_tag2.

[0184] In some examples, after the classification module generates the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected, and the portrait module generates the portrait information corresponding to the second image to be detected, the gallery APP can obtain the category information corresponding to the first image to be detected, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected from the classification module and the portrait module. Then, according to the category information corresponding to the first image to be detected, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected, all the images to be detected in the gallery APP are classified to obtain multiple classification modules. For example, multiple classification modules include portraits, locations, and affairs, etc., and affairs can be further classified into scenery, food, animals, sports, transportation, documents, etc. Afterwards, when the user uses the gallery APP, the image type can be quickly located based on these classification modules to find the required media data.

[0185] For example, Figure 6 The classification interface of a gallery APP is shown as an example, and the classification interface includes portrait classification, location classification and business classification. There are more detailed sub-categories under the location classification, such as Beijing sub-categories, Shanghai sub-categories, Shenzhen sub-categories and Changsha sub-categories. There are more detailed sub-categories under the business classification, such as food sub-categories, scenery sub-categories, document sub-categories and sports sub-categories.

[0186] In some examples, the gallery APP is also provided with a collection function and a sharing function. When a user uses the collection function in the gallery APP, the gallery APP can generate a collection album corresponding to the collection function. When a user uses the sharing function in the gallery APP, the gallery APP can generate a sharing album corresponding to the collection function.

[0187] Moreover, the Gallery APP also has the functions of recording and capturing screenshots, deleting, downloading, etc. Based on these functions, corresponding albums can be created. For example, recording and capturing albums, deleting albums, and downloading albums, etc. With the development of the technology of the Gallery APP, the functions of the Gallery APP will become more diversified. Therefore, you can also refer to this method to generate albums corresponding to various functions, and this disclosure does not limit this.

[0188] For example, Figure 7 An exemplary display interface of all photos in a gallery APP is shown, and the display interface includes multiple album types. For example, the camera album, all photo album, video album, screen capture album, favorite album, shared album and more albums. More albums include downloaded albums, other albums and recently deleted albums.

[0189] In addition, the gallery APP can also record the media data in the collection album to generate collection information. The media data in the shared album is recorded to generate sharing information. Among them, the collection information includes the serial number, image id, image identifier and image collection marker. The sharing information includes the serial number, image id, image identifier and image sharing marker. Afterwards, the gallery APP can send the collection information and sharing information to the gallery database, and the gallery database can generate the collection database and sharing database based on the collection information and sharing information, and then synchronize the collection database and sharing database to the media data middle platform APP, so that when the media data middle platform APP receives the media data query request of the APP to be processed, it can quickly feedback the query results of the media data to the APP to be processed based on the collection database and sharing database.

[0190] Step 508: The portrait module sends the portrait information corresponding to the second image to be detected to the image library database.

[0191] After the portrait module obtains the facial information corresponding to the second image to be detected, the facial information corresponding to the second image to be detected can be sent to the gallery database. Afterwards, the media data middle platform APP obtains the facial information corresponding to the second image to be detected through the gallery database, and provides the facial information corresponding to the second image to be detected to the APP to be processed. The APP to be processed can display the second image to be detected to the user according to the facial information corresponding to the second image to be detected, so as to facilitate the user to quickly find the required media data and improve data search efficiency.

[0192] Step 509: The image library database receives the portrait information corresponding to the second image to be detected, and generates a portrait database based on the portrait information corresponding to the second image to be detected.

[0193] After the gallery database receives the portrait information corresponding to the second image to be detected, the gallery database can generate a portrait database based on the portrait information corresponding to the second image to be detected. Then the portrait database is synchronized to the media data middle station APP, so that when the media data middle station APP receives the media data query request of the APP to be processed, it can quickly feedback the query result of the media data to the APP to be processed based on the portrait database.

[0194] The portrait database is used to characterize the correspondence between the second image to be detected and the portrait in the second image to be detected. Exemplarily, the portrait database can describe the correspondence between the first face image to be detected and the portrait in the second image to be detected in the form of a table.

[0195] The portrait database may include a portrait data table and a portrait statistics table, wherein the portrait data table may be a merge_face table and the portrait statistics table may be a merge_tag table.

[0196] The portrait data table is used to represent the facial information corresponding to all the images to be detected in the electronic device. The portrait data table may include multiple sub-portrait data information. Each sub-face data information includes at least a sequence number, an image address of the image to be detected, an image identifier of the image to be detected, a portrait album identifier information corresponding to the image to be detected, the position information of each portrait in the image to be detected in the image to be detected, and the portrait similarity of each portrait in the image to be detected, etc.

[0197] It can be understood that since the portrait data table includes the facial information corresponding to all images to be detected in the electronic device, when there is no face in the image to be detected, the portrait album identification information corresponding to the image to be detected, the position information of each portrait in the image to be detected in the image to be detected, and the portrait similarity of each portrait in the image to be detected can be represented by empty characters, 0 characters, or -1 characters, etc., and the present disclosure does not impose any restrictions on this.

[0198] The portrait statistics table is used to characterize the portrait information data corresponding to the second image to be detected in the electronic device. The portrait data table may include multiple sub-portrait statistical data. Each portrait data includes at least the portrait album identification information corresponding to the image to be detected and the identification information corresponding to each portrait in the image to be detected.

[0199] Step 510, the search module of the gallery APP obtains the category information corresponding to the first image to be detected, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected, and constructs index data based on the category information corresponding to the first image to be detected, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected, and sends the index data to the search service.

[0200] The classification information corresponding to the first image to be detected includes category information corresponding to the first image to be detected, location information corresponding to the first image to be detected, and portrait information corresponding to the second image to be detected.

[0201] In some examples, the search module of the gallery APP is used to respond to the user's search request and feedback the search results corresponding to the search request.

[0202] For example, a gallery application is running in the foreground of the electronic device, and the gallery application displays Figure 8 The interface shown in (a) is as follows: Figure 8The interface shown in (a) may include pictures, videos, and function buttons, etc. The function buttons may include: photo button 801, album button 802, moment button 803, creation button 804, search button 805, and more function button 806; search button 805 may be used to search for target images according to search terms. The user can use the search function of the gallery application by clicking the search button 805. When the electronic device receives a trigger operation for the search button 805, the electronic device may enter the following Figure 8 The interface shown in (b).

[0203] exist Figure 8 In the interface shown in (b) of FIG. 1 , the electronic device may display an image search bar 807. In one possible implementation, the user may input a search term in text in the search bar 807; in another possible implementation, the search bar 807 may include a voice button 808; the user may input a search term in the search bar 807 by voice based on the voice button 808.

[0204] When the electronic device receives a search term input or entered by the user in the search bar, the electronic device may display multiple search results related to the search term according to the search term; wherein the search results may be pictures and / or videos related to the search term, and the number of images in the search results may be one or more. For example, with the search term "Beijing", the electronic device displays the search results. The search results include search prompt information 809, search picture information 810, and searched pictures 811. Among them, the search prompt information 809 displays the "location" identifier and "Beijing City". The search picture information 810 displays "XX pictures found".

[0205] In some examples, the search module of the gallery APP can also be used to build index data and send the index data to the search service.

[0206] The process of the search module constructing index data and sending the index data to the search service is as follows: after the classification module generates the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected, the classification module can send the category information corresponding to the first image to be detected and the location information corresponding to the first image to be detected to the search module. After the portrait module generates the portrait information corresponding to the second image to be detected, the portrait module can also send the portrait information corresponding to the second image to be detected to the search module.

[0207] After the search module receives the category information corresponding to the first image to be detected, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected, it can construct index data based on the category information corresponding to the first image to be detected, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected, and send the index data to the search service, so that the subsequent search service can respond to the user's search operation and provide search results to the user.

[0208] The index data may include the image ID of the image to be detected, the image identification of the image to be detected, the classification label corresponding to the image to be detected, the classification probability corresponding to the classification label, the image longitude of the image to be detected, the image latitude of the image to be detected, the location major classification corresponding to the image to be detected, the location minor classification corresponding to the first image to be detected, the portrait album identification information of the image to be detected, the feature data of the portrait in the image to be detected, the position of each portrait in the image to be detected in the image to be detected, and the portrait similarity of each portrait in the image to be detected, etc. These index data are used to describe the content and features of the image to be detected, so as to facilitate subsequent search and query operations.

[0209] In other examples, the process of the search module constructing index data and sending the index data to the search service is as follows: after the gallery database constructs the location database, the transaction database, the portrait database, the collection database, and the sharing database, the gallery database can send the location database, the transaction database, the portrait database, the collection database, and the sharing database to the search module. The search module can construct index data based on the location database, the transaction database, the portrait database, the collection database, and the sharing database, and send the index data to the search service.

[0210] The present application does not limit the way in which the search module obtains the processed data (i.e., the category information corresponding to the first image to be detected, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected, or the location database, transaction database, portrait database, collection database, and sharing database).

[0211] Step 511: The search service receives and stores index data.

[0212] After the search module builds the index data, the search module can send the index data to the search service. After receiving the index data, the search service can store the index data in the search service. Later, when the gallery APP or the media data middle platform APP has a search demand, the gallery APP or the media data middle platform APP can feedback the search results to the user based on the search service, so that the user can quickly filter out the required media data.

[0213] It should be noted that the embodiment of the present disclosure does not limit the execution order of the above steps 506, 508-509 and 510. For example, step 506, step 508-509 may be executed first, and then step 510; step 510 may be executed first, and then step 506, step 508-509; or step 506, step 508-509 and step 510 may be executed simultaneously, which may be determined according to actual usage requirements.

[0214] Step 512: In the target case, the metadata management module in the media data middle platform APP sends an update request to the gallery database.

[0215] The update request is used to request updated media data in the gallery database.

[0216] In the target case, the metadata management module in the media data middle platform APP will send an update request to the gallery database to timely obtain the updated media data in the gallery database (i.e., location database, transaction database, portrait database, collection database, and sharing database), so as to facilitate subsequent response to user operations and quickly provide users with the required media data.

[0217] In some examples, the target situation is: when the media data middle platform APP is in operation, if the metadata management module in the media data middle platform APP detects that the data in the gallery database has changed, the metadata management module sends an update request to the gallery database.

[0218] In other examples, the target situation is: the media data middle platform APP is not in running state. When other applications pull up the media data middle platform APP, the metadata management module in the media data middle platform APP sends an update request to the gallery database.

[0219] After the metadata management module sends an update request to the gallery database, the updated media data of the location database, transaction database, portrait database, collection database and sharing database in the gallery database can be obtained through the update request. In this way, the media data middle station APP can obtain all the media data in the electronic device. Later, when the APP to be processed attempts to obtain the media data required by the user through the media data middle station APP, the media data middle station APP can provide the required media data to the APP to be processed in a timely manner. This avoids the situation where the APP to be processed cannot obtain the required media data due to untimely updates.

[0220] Step 513: In response to the update request, the gallery database sends the updated media data to the metadata management module in the media data middle platform APP.

[0221] In some examples, when the metadata management module in the media data middle station APP monitors that the data in the gallery database has changed, it will send an update request to the gallery database. At this time, the gallery database will send all the updated media data to the metadata management module in the media data middle station APP. Alternatively, when the media data middle station APP is pulled up by other applications, the metadata management module in the media data middle station APP will send an update request to the gallery database. At this time, the gallery database can first combine the media data sent last time to determine the data that needs to be sent to the media data middle station APP this time. Then, in response to the update request, send the updated media data to the media data middle station APP.

[0222] Step 514: The metadata management module receives and stores updated media data from the gallery database.

[0223] After receiving the updated media data from the gallery database, the metadata management module can store the updated media data. Later, when the thumbnail management module and search module in the media data platform APP need media data, the required media data can be obtained through the metadata management module.

[0224] After the media data in the gallery database is stored in the metadata management module in the media data middle platform APP, if the user needs to access the media data in the gallery APP through the pending APP, the pending APP can provide it to the user through the metadata management module in the media data middle platform APP. And because the location database, transaction database, portrait database, collection database and sharing database in the gallery database all record the category information of the media data in the gallery APP in detail, this is equivalent to the media data middle platform APP also providing the pending APP with the category information of the media data in the gallery APP. This makes it easier for the pending APP to find and obtain the required media data.

[0225] Step 515: The APP to be processed receives a portrait selection operation from the user, and sends a portrait album query request to the metadata management module in response to the portrait selection operation.

[0226] The portrait selection operation may specifically be an operation triggered by a user on a portrait button in the display interface of the APP to be processed. The portrait album query request is used to request to obtain at least one portrait album mark information. The operation triggered by a user on a portrait button in the display interface of the APP to be processed may be any one of a long press operation, a single click operation, a double click operation, a triple click operation, a knuckle tapping operation, and a multi-finger selection operation.

[0227] Combining steps 513 and 514, it can be seen that the metadata management module can obtain all the media data in the gallery database, and the gallery database stores all the media data of the gallery APP. In other words, the media data middle platform APP can obtain all the media data of the gallery APP. Therefore, when other APPs (for example, APPs to be processed) cannot directly access the media data in the gallery APP, but need to access the media data in the gallery APP, they can obtain the media data in the gallery APP by accessing the media data middle platform APP. In this process, the media data middle platform APP is equivalent to a bridge for data interaction between the gallery APP and other APPs. In this way, other APPs can obtain the media data in the gallery application.

[0228] Moreover, what is stored in the gallery database is the classification data of all media data in the gallery APP (for example, location database, transaction database, portrait database, collection database, and sharing database), so the media data middle platform APP also obtains the classification data of the media data in the gallery APP. Correspondingly, the media data middle platform APP can provide the classification data of the media data in the gallery APP for the APP to be processed.

[0229] When a user accesses media data in the gallery APP through the pending APP, the pending APP can display the media data in the gallery APP to the user in the form of classified data. In this way, the user can quickly locate the required media data from a large amount of media data, thereby improving data search efficiency. In addition, the pending APP displays the media data in the gallery APP to the user in the form of classified data, which also provides a new display form.

[0230] The APP to be processed can obtain the classification data of all media data in the gallery APP and / or the category information corresponding to the first image to be detected generated by the gallery APP, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected through the media data middle station APP. Then, based on the classification data of all media data in the gallery APP and / or the category information corresponding to the first image to be detected generated by the gallery APP, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected, a plurality of function buttons can be set in the display interface of the APP to be processed, and each function button corresponds to a classification option. For example, a plurality of function buttons may include all buttons, portrait buttons, location buttons, business buttons, collection buttons, and search buttons, etc., that is, the display interface of the APP to be processed sets all options, portrait options, location options, business options, collection options, and search options. The present disclosure does not limit the number and setting method of the function buttons.

[0231] When the user uses the APP to be processed, since multiple function buttons (i.e., multiple classification options) are designed in the display interface of the APP to be processed, the user can directly click on one of the multiple function buttons to find the required media data. At this time, the APP to be processed can receive the user's click operation on any of the above multiple function buttons (i.e., all buttons, portrait buttons, location buttons, transaction buttons, favorite buttons, and search buttons, etc.), thereby sending a corresponding query request to the metadata management module. Exemplarily, the APP to be processed receives the user's selection operation on the portrait button (i.e., the portrait option), and in response to the user's selection operation on the portrait button, the APP to be processed sends a portrait album query request to the metadata management module.

[0232] Step 516: The metadata management module receives a portrait album query request.

[0233] Step 517: In response to the portrait album query request, the metadata management module obtains the portrait album identification information and sends the portrait album identification information to the APP to be processed.

[0234] After the metadata management module receives the portrait album query request of the APP to be processed, the metadata management module searches for at least one portrait album identification information in the portrait database stored in the gallery database, and sends the found portrait album identification information to the APP to be processed.

[0235] Since the portrait database obtained by the metadata management module from the gallery database may include one or more portrait album identification information, after the metadata management module obtains the portrait album identification information, all portrait album identification information needs to be sent to the processing APP.

[0236] Step 518: The APP to be processed receives the portrait album identification information and sends a portrait album thumbnail request to the thumbnail management module.

[0237] The portrait album thumbnail request is used to request the thumbnail management module to generate multiple portrait album thumbnails based on the portrait album identification information. The portrait album thumbnail request includes the portrait album identification information.

[0238] After the APP to be processed receives the portrait album identification information, the APP to be processed may send a portrait album thumbnail request to the thumbnail management module, so that the thumbnail management module generates multiple portrait album thumbnails according to the portrait album identification information, and displays the multiple portrait album thumbnails in the display interface of the APP to be processed.

[0239] Step 519: The thumbnail management module receives the portrait album thumbnail request and generates a portrait album thumbnail corresponding to the portrait album identification information based on the portrait album identification information.

[0240] In some examples, after receiving a portrait album thumbnail request, the thumbnail management module can determine the number of portrait album identification information based on the portrait album thumbnail request, and generate one portrait album thumbnail if there is only one portrait album identification information. If there are multiple portrait album identification information, multiple portrait album thumbnails are generated.

[0241] Since the portrait album identification information corresponds to a portrait album set, and a portrait album set includes at least one image to be detected, the thumbnail management module can select an image to be detected from the album set as the cover image of the portrait album thumbnail based on the album set. Then, the selected cover image is cropped and adjusted as needed to generate a portrait album thumbnail, and finally, multiple portrait album thumbnails are saved and exported for use or display by the APP to be processed.

[0242] Step 520: The thumbnail management module sends the portrait album thumbnail corresponding to the portrait album identification information to the APP to be processed.

[0243] Before the thumbnail management module sends the portrait album thumbnail corresponding to the portrait album identification information to the APP to be processed, it is necessary to further determine whether the size of the portrait album thumbnail meets the restrictions on data exchange between processes. If it does, the portrait album thumbnail corresponding to the portrait album identification information is directly sent to the APP to be processed. If it does not meet the requirements, the portrait album thumbnail corresponding to the portrait album identification information can be scaled and then sent to the APP to be processed.

[0244] Step 521, the APP to be processed receives the portrait album thumbnail corresponding to the portrait album identification information, and displays the portrait album thumbnail corresponding to the portrait album identification information.

[0245] After the APP to be processed receives the portrait album thumbnail corresponding to the portrait album identification information sent by the thumbnail management module, the portrait album thumbnail corresponding to the portrait album identification information can be displayed on the display interface of the APP to be processed. A portrait album thumbnail corresponds to a portrait, and a portrait album thumbnail can be considered as a portrait album thumbnail set. Multiple thumbnails in the portrait album thumbnail all contain the portrait.

[0246] For example, the pending APP displays as follows Fig. 9 The interface shown in (a) is as follows: Fig. 9The interface shown in (a) in the figure may include multiple function buttons. The function buttons may include: all buttons 901, portrait buttons 902, location buttons 903, business buttons 904, favorite buttons 905, and group photo buttons 906. The portrait button 902 may be used to provide all media data related to portraits in the electronic device. After the user clicks the portrait button, the APP to be processed may respond to the user's click operation on the portrait button, first obtain the portrait album identification information from the metadata management module, then obtain the portrait album thumbnail corresponding to the portrait album identification information from the thumbnail management module, and display multiple portrait album thumbnails 907. For example, the portrait album thumbnails include thumbnails corresponding to portrait A, thumbnails corresponding to portrait B, thumbnails corresponding to portrait C, thumbnails corresponding to portrait D, thumbnails corresponding to portrait E, and thumbnails corresponding to portrait F.

[0247] Step 522: In response to the user selecting the first portrait album thumbnail in the portrait album thumbnails, the APP to be processed sends a request to obtain the first portrait information to the metadata management module.

[0248] The first portrait information acquisition request is used to request to acquire image information corresponding to the first portrait. The image information corresponding to the first portrait includes at least the number of images corresponding to the first portrait, the image path of each image corresponding to the first portrait, and the image name of each image corresponding to the first portrait.

[0249] After the display interface of the APP to be processed displays the portrait album thumbnails, the user can select a portrait album thumbnail (i.e., the first portrait album thumbnail) from the portrait album thumbnails based on the needs. After the APP to be processed detects the user's selection operation on the first portrait album thumbnail, the APP to be processed can send a first portrait information acquisition request to the metadata management module to obtain the image information corresponding to the first portrait, so as to facilitate the subsequent display of the thumbnail corresponding to the first portrait to the user.

[0250] Step 523: The metadata management module receives a first portrait information acquisition request.

[0251] Step 524: In response to the first portrait information acquisition request, the metadata management module acquires the image information corresponding to the first portrait, and sends the image information corresponding to the first portrait to the APP to be processed.

[0252] Since the metadata management module has pre-synchronized the media data of the gallery database, after the metadata management module receives the request to obtain the first portrait information, the metadata management module can respond to the request to obtain the first portrait information, search for image information corresponding to the first portrait in the portrait database in the gallery database, and send the image information corresponding to the first portrait to the APP to be processed, so that the APP to be processed can display the thumbnail corresponding to the first portrait to the user based on the image information corresponding to the first portrait.

[0253] Step 525: The APP to be processed receives the image information corresponding to the first portrait, and sends a first portrait thumbnail acquisition request to the thumbnail management module.

[0254] The first person portrait thumbnail acquisition request is used to request to acquire at least one thumbnail corresponding to the first person portrait. The first person portrait thumbnail acquisition request includes image information corresponding to the first person portrait.

[0255] After the APP to be processed obtains the image information corresponding to the first portrait from the metadata management module, it can send a first portrait thumbnail acquisition request to the thumbnail management module, so that the thumbnail management module generates a thumbnail of each image corresponding to the first portrait based on the image information corresponding to the first portrait in the first portrait thumbnail acquisition request.

[0256] Step 526: The thumbnail management module receives a request for obtaining a first person portrait thumbnail.

[0257] Step 527, in response to the first portrait thumbnail acquisition request, the thumbnail management module generates a thumbnail of each image corresponding to the first portrait based on the image information corresponding to the first portrait in the first portrait thumbnail acquisition request, and sends the thumbnail of each image corresponding to the first portrait to the APP to be processed.

[0258] After the thumbnail management module receives the first person portrait thumbnail acquisition request, the thumbnail management module can sequentially generate thumbnails of each image corresponding to the first person portrait based on the image information corresponding to the first person portrait carried in the first person portrait thumbnail acquisition request. After generating the thumbnail of each image corresponding to the first person portrait, the thumbnail of each image corresponding to the first person portrait can be sent to the APP to be processed, so that the APP to be processed displays the thumbnail of each image corresponding to the first person portrait to the user.

[0259] In some examples, the thumbnail management module can generate a thumbnail of each image corresponding to the first portrait in sequence based on the image information corresponding to the first portrait carried in the first portrait thumbnail acquisition request. The process is as follows: the thumbnail management module obtains the number of images corresponding to the first portrait, the image path of each image corresponding to the first portrait, and the picture name of each image corresponding to the first portrait in the image information corresponding to the first portrait, and then traverses each image corresponding to the first portrait. For each image corresponding to the first portrait, use an image processing library or a thumbnail tool to proportionally scale or crop each image corresponding to the first portrait according to preset thumbnail size parameters to generate a thumbnail of each image corresponding to the first portrait.

[0260] In some examples, when data needs to be transferred between two processes of the Android system, the data transfer restriction conditions need to be strictly followed. Exemplarily, the data transfer restriction conditions between the two processes include that the data transfer size between the two processes cannot exceed 1016kb.

[0261] For some ultra-high-definition images, the size of the thumbnail file processed by the thumbnail management module may still exceed 1M (i.e. 1024kb). Based on the aforementioned data transmission restrictions, if the thumbnail file corresponding to the ultra-high-definition image is to be transferred to other processes, the thumbnail file corresponding to the ultra-high-definition image needs to be further processed, otherwise the thumbnail file corresponding to the ultra-high-definition image cannot be transferred to other processes.

[0262] Based on the above possible scenarios, before the thumbnail management module sends the thumbnail of each image corresponding to the first portrait to the APP to be processed, the thumbnail management module also needs to determine whether the size of the thumbnail of each image corresponding to the first portrait exceeds the preset threshold. In the case where the size of the thumbnail of each image corresponding to the first portrait exceeds the preset threshold, it is also necessary to scale the thumbnail of each image corresponding to the first portrait so that the size of the thumbnail of each image corresponding to the scaled first portrait does not exceed the preset threshold. In the case where the size of the thumbnail of each image corresponding to the first portrait does not exceed the preset threshold, the thumbnail management module can directly send the thumbnail of each image corresponding to the first portrait to the APP to be processed. In order to avoid abnormal data interaction between threads other than thumbnail files, exemplarily, the preset threshold is set to 500Kb. The preset threshold is set to 500Kb to ensure that the data interaction between processes is normal as much as possible, and clearer thumbnails can be transmitted between processes, thereby avoiding the problem of unclear images caused by too small thumbnails.

[0263] In some examples, the specific process of scaling the thumbnail of each image corresponding to the first portrait may be to directly take half the size of the thumbnail of each image corresponding to the first portrait to obtain the scaled thumbnail of each image corresponding to the first portrait. Then continue to determine whether the size of the thumbnail of each image corresponding to the scaled first portrait exceeds a preset threshold. If not, directly send the scaled thumbnail of each image corresponding to the first portrait to the APP to be processed. If it exceeds, continue to take half the size of the thumbnail of each image corresponding to the scaled first portrait until the size of the thumbnail of each image corresponding to the first portrait does not exceed the preset threshold.

[0264] Step 528: The APP to be processed receives the thumbnail of each image corresponding to the first portrait, and displays the thumbnail of each image corresponding to the first portrait.

[0265] After the APP to be processed receives the thumbnail of each image corresponding to the first person portrait, the APP to be processed will display the thumbnail of each image corresponding to the first person portrait on the display interface. When the user views the thumbnail of each image corresponding to the first person portrait on the display interface of the APP to be processed, the user can perform functional operations such as editing and sharing on one or more thumbnails of each image corresponding to the first person portrait based on needs. The functional operations that the user can perform are related to the functions that the APP to be processed can provide, and this disclosure does not limit this.

[0266] For example, Fig. 9 (a) in the figure shows a plurality of portrait album thumbnails 907. In response to the user selecting a thumbnail corresponding to portrait A from the plurality of portrait album thumbnails 907, the APP to be processed displays the following Fig. 9 The interface shown in (b). Fig. 9 (b) in the figure shows an image interface 908 of the portrait A. The image interface 908 of the portrait A may include multiple photos of the portrait A, and each photo of the portrait A includes at least the portrait A.

[0267] Step 529: In response to the user's selection operation on the second portrait album thumbnail from the multiple portrait album thumbnails, the APP to be processed sends a request to obtain the second portrait information to the metadata management module.

[0268] The second portrait may include at least two portraits. The second portrait album thumbnail is at least two portrait album thumbnails. The second portrait information acquisition request is used to request to acquire image information corresponding to the second portrait.

[0269] The image information corresponding to the second portrait includes at least the number of images corresponding to the second portrait, the image path of each image corresponding to the second portrait, and the image name of each image corresponding to the second portrait.

[0270] The number of images corresponding to the second portrait refers to the number of images corresponding to each of the at least two portraits. The image path of each image corresponding to the second portrait refers to the image path of each image corresponding to each of the at least two portraits. The image name of each image corresponding to the second portrait refers to the image number of each image corresponding to each of the at least two portraits.

[0271] When the user performs a long press operation on a portrait album thumbnail in the display interface of the pending APP, the portrait album multi-selection function can be triggered. The portrait album multi-selection function is specifically: in the portrait album thumbnail, the user can select at least two portrait album thumbnails.

[0272] Therefore, in the portrait album thumbnails, the user triggers the selection operation of the second portrait album thumbnail specifically as follows: after the user performs a long press operation on a portrait album thumbnail in the display interface of the pending APP, the user can continue to select other portrait album thumbnails in the portrait album thumbnails until the user's selection is completed. In this process, all portrait album thumbnails selected by the user are second portrait album thumbnails.

[0273] When the APP to be processed detects the user's selection operation on the second portrait album thumbnail, the APP to be processed can send a second portrait information acquisition request to the metadata management module to obtain the image information corresponding to the second portrait, so as to facilitate the subsequent display of the thumbnail corresponding to the second portrait to the user.

[0274] Step 530: The metadata management module receives a second portrait information acquisition request.

[0275] Step 531: In response to the second portrait information acquisition request, the metadata management module acquires the image information corresponding to the second portrait, and sends the image information corresponding to the second portrait to the APP to be processed.

[0276] Since the metadata management module has pre-synchronized the media data of the gallery database, after the metadata management module receives the request to obtain the second portrait information, the metadata management module can respond to the request to obtain the second portrait information, search for image information corresponding to the second portrait in the portrait database in the gallery database, and send the image information corresponding to the second portrait to the APP to be processed, so that the APP to be processed can display the thumbnail corresponding to the second portrait to the user based on the image information corresponding to the second portrait.

[0277] In some examples, the second portrait includes at least two portraits, for example, the second portrait includes portrait 1 and portrait 2. Since the metadata management module pre-synchronizes the media data of the gallery database, the process of the metadata management module obtaining the image information corresponding to the second portrait can be: in response to the user's selection operation of the portrait button corresponding to portrait 1 (i.e., portrait 1 option) and the portrait button corresponding to portrait 2 (i.e., portrait 2 option), the APP to be processed sends a request to obtain the second portrait information to the metadata management module. In response to the request to obtain the second portrait information, the metadata management module first obtains the face album identification information of portrait 1 (i.e., group_tag of portrait 1) from the portrait database based on the portrait database in the gallery database and the image information corresponding to portrait 1 in the image information corresponding to the second portrait. Then, all the images to be detected corresponding to the group_tag of portrait 1 are searched in the portrait database, and all the images to be detected corresponding to the group_tag of portrait 1 can be considered as the first portrait album set.

[0278] Next, determine whether there is a second portrait album set corresponding to the face album identification information of portrait 2 (i.e., group_tag of portrait 2) in the first portrait album set. If so, it means that the metadata management module includes a group photo of portrait 1 and portrait 2, and the group photo of portrait 1 and portrait 2 includes at least portrait 1 and portrait 2. If not, it means that the metadata management module does not include a group photo of portrait 1 and portrait 2. After determining that the metadata management module includes a group photo of portrait 1 and portrait 2, it means that the second image set is a group photo set of portrait 1 and portrait 2. Finally, based on the portrait database, determine the image information corresponding to the second portrait album set, and the image information corresponding to the second portrait album set is the image information corresponding to the second portrait.

[0279] Step 532: The APP to be processed receives the image information corresponding to the second portrait, and sends a request for obtaining the thumbnail of the second portrait to the thumbnail management module.

[0280] The second person portrait thumbnail acquisition request is used to request to acquire at least one thumbnail corresponding to the second person portrait. The second person portrait thumbnail acquisition request includes image information corresponding to the second person portrait.

[0281] After the APP to be processed obtains the image information corresponding to the second portrait from the metadata management module, it can send a second portrait thumbnail acquisition request to the thumbnail management module, so that the thumbnail management module generates a thumbnail of each image corresponding to the second portrait based on the image information corresponding to the second portrait in the second portrait thumbnail acquisition request.

[0282] Step 533: The thumbnail management module receives a request for obtaining a second portrait thumbnail.

[0283] Step 534, in response to the second portrait thumbnail acquisition request, the thumbnail management module generates a thumbnail of each image corresponding to the second portrait based on the image information corresponding to the second portrait in the second portrait thumbnail acquisition request, and sends the thumbnail of each image corresponding to the second portrait to the APP to be processed.

[0284] After the thumbnail management module receives the second portrait thumbnail acquisition request, the thumbnail management module can sequentially generate thumbnails of each image corresponding to the second portrait based on the image information corresponding to the second portrait carried in the second portrait thumbnail acquisition request. After generating the thumbnail of each image corresponding to the second portrait, the thumbnail of each image corresponding to the second portrait can be sent to the APP to be processed, so that the APP to be processed displays the thumbnail of each image corresponding to the second portrait to the user.

[0285] It can be understood that the process by which the thumbnail management module generates a thumbnail of each image corresponding to the second portrait based on the image information corresponding to the second portrait in the second portrait thumbnail acquisition request is similar to the process by which the thumbnail management module in the above step 527 generates a thumbnail of each image corresponding to the first portrait in sequence based on the image information corresponding to the first portrait in the first portrait thumbnail acquisition request, and will not be repeated here.

[0286] In some examples, combined with step 527, it can be seen that before the thumbnail management module sends the thumbnail of each image corresponding to the second portrait to the APP to be processed, the thumbnail management module also needs to determine whether the size of the thumbnail of each image corresponding to the second portrait exceeds the preset threshold. In the case where the size of the thumbnail of each image corresponding to the second portrait exceeds the preset threshold, it is also necessary to scale the thumbnail of each image corresponding to the second portrait so that the size of the scaled thumbnail of each image corresponding to the second portrait does not exceed the preset threshold. In the case where the size of the thumbnail of each image corresponding to the second portrait does not exceed the preset threshold, the thumbnail management module can directly send the thumbnail of each image corresponding to the second portrait to the APP to be processed.

[0287] Among them, the specific process of scaling the thumbnail of each image corresponding to the second portrait can be to directly take half of the size of the thumbnail of each image corresponding to the second portrait to obtain the scaled thumbnail of each image corresponding to the second portrait. Then continue to determine whether the size of the thumbnail of each image corresponding to the scaled second portrait exceeds the preset threshold. If not, directly send the thumbnail of each image corresponding to the scaled second portrait to the APP to be processed. If it exceeds, continue to take half of the size of the thumbnail of each image corresponding to the scaled second portrait until the size of the thumbnail of each image corresponding to the final second portrait does not exceed the preset threshold.

[0288] Step 535 , the APP to be processed receives the thumbnail of each image corresponding to the second portrait, and displays the thumbnail of each image corresponding to the second portrait.

[0289] After the APP to be processed receives the thumbnail of each image corresponding to the second portrait, the APP to be processed will display the thumbnail of each image corresponding to the second portrait. When the user views the thumbnail of each image corresponding to the second portrait, the user can perform functional operations such as editing and sharing on one or more thumbnails of each image corresponding to the second portrait based on needs. The functional operations that the user can perform are related to the functions that the APP to be processed can provide, and the present disclosure does not limit this.

[0290] For example, Fig. 9 (a) in the figure shows the portrait album thumbnail 907. In response to the user's selection operation of the thumbnail corresponding to the portrait B, the thumbnail corresponding to the portrait C, and the thumbnail corresponding to the portrait D in the portrait album thumbnail 907, the APP to be processed displays the following Fig. 9 The interface shown in (c) is as follows: Fig. 9 In (c), the thumbnails corresponding to the portrait B, the thumbnails corresponding to the portrait C, and the thumbnails corresponding to the portrait D are all marked with a “√”. The “√” mark indicates that the user has selected the thumbnails corresponding to the portrait B, the thumbnails corresponding to the portrait C, and the thumbnails corresponding to the portrait D in the portrait album thumbnails 907. Fig. 9 In the interface shown in (c) of FIG. 1 , for the selection operation of the “photo button” 909, the APP to be processed displays the following Fig. 9 The interface shown in (d) is as follows: Fig. 9 (d) in the figure shows a group photo interface 910 of portraits B, C and D. The group photo interface 910 of portraits B, C and D may include multiple group photo images, and each group photo image includes at least portraits B, C and D.

[0291] The user's selection operation on the thumbnail corresponding to the portrait B, the thumbnail corresponding to the portrait C, and the thumbnail corresponding to the portrait D may be performed by the user clicking on the thumbnail corresponding to the portrait B, the thumbnail corresponding to the portrait C, and the thumbnail corresponding to the portrait D in sequence. Alternatively, the user may slide to the thumbnail corresponding to the portrait B, the thumbnail corresponding to the portrait C, and the thumbnail corresponding to the portrait D in sequence, and the present disclosure does not limit this.

[0292] In some examples, it can be seen from step 515 that the display interface of the APP to be processed can be designed with multiple function buttons, for example, the multiple function buttons can include an all button, a portrait button, a location button, a transaction button, a favorite button, and a search button, etc. The user can select any one of the all button, the portrait button, the location button, the transaction button, the favorite button, or the search button based on the needs.

[0293] Referring to the implementation process of step 515 to step 535, if the APP to be processed receives a user's click operation on the location button 904, the APP to be processed displays the following Fig.10 The interface shown in (a). Fig.10 (a) in the figure shows a plurality of location album thumbnails 1001. For example, the plurality of location album thumbnails 1001 include thumbnails corresponding to Beijing, Shanghai, Shenzhen, Changsha, Suzhou, and Tianjin.

[0294] In response to the user selecting the thumbnail corresponding to Beijing from the multiple location album thumbnails 1001, the processing APP displays the following Fig.10 The interface shown in (b). Fig.10 (b) in the figure shows an image interface 902 corresponding to Beijing City. The image interface 1002 corresponding to Beijing City may include multiple photos of Beijing City.

[0295] If the pending APP receives a user click operation on the transaction button 905, the pending APP displays the following Fig.10 The interface shown in (c). Fig.10 (c) in the figure shows a plurality of business album thumbnails 1003. For example, the plurality of business album thumbnails 1003 include thumbnails corresponding to food, thumbnails corresponding to scenery, thumbnails corresponding to documents, thumbnails corresponding to sports, thumbnails corresponding to vehicles, and thumbnails corresponding to buildings.

[0296] In response to the user's selection operation of a thumbnail corresponding to a food in the multiple classified album thumbnails 1003, the processing APP displays the following Fig.10 The interface shown in (d). Fig.10 (d) in the figure shows an image interface 1004 corresponding to the food, and the image interface 1004 corresponding to the food may include a plurality of photos related to the food.

[0297] If the APP to be processed receives a user click operation on the collection button 903, the APP to be processed displays the following Fig.10 The interface shown in (e). Fig.10 (c) in the figure shows at least one favorite media data 1005. The at least one favorite media data 1005 may be an image or a video, which is not limited in the present disclosure.

[0298] Step 536: The APP to be processed receives the search information written by the user in the search box of the search interface of the APP to be processed.

[0299] The search information includes search terms.

[0300] In combination with the above, it can be known that the APP to be processed can provide multiple function buttons for users, including a search button. Users can click the search button based on their needs to enter the search interface. Generally, the search interface includes a search box. Users can write search information in the search bar in a variety of ways.

[0301] In some examples, the search term can be entered in text form or in voice form. Figure 8 In (b), the user can enter the search term in the search bar in the form of text, or use the voice button in the search bar to enter the search term in the form of voice. This application does not limit the way of writing the search term.

[0302] Step 537: In response to the search operation triggered by the user in the search interface, the APP to be processed sends a search query request to the search management module of the media data middle platform APP.

[0303] After the user enters the search information in the search box, the APP to be processed can receive the search operation from the user. At this time, the APP to be processed can send a search query request to the search management module of the media data middle platform APP to obtain the search results corresponding to the search information (that is, the media data corresponding to the search term, which can be at least one of the multiple images to be detected).

[0304] The search query request is used to request to obtain search results corresponding to the search information. The search query request includes the search information. Exemplarily, the search operation can be any one of a long press operation, a single click operation, a double click operation, a triple click operation, a knuckle tapping operation, and a multi-finger selection operation.

[0305] In combination with step 510 and step 511, it can be seen that after the search module in the gallery APP obtains the category information corresponding to the first image to be detected, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected, the search module can build index data based on the category information corresponding to the first image to be detected, the location information corresponding to the first image to be detected, and the portrait information corresponding to the second image to be detected, and store the index data in the search service. Therefore, when the APP to be processed receives a search operation from the user, the APP to be processed can use the search service to display the search results corresponding to the search operation to the user.

[0306] Since the APP to be processed cannot directly access the search service, the media data middle platform APP can access the search service, so the APP to be processed can send a search query request to the search management module in the media data middle platform APP to use the search management module to obtain the search results corresponding to the search selection operation.

[0307] Step 538: The search management module receives the search query request.

[0308] Step 539: In response to the search query request, the search management module sends a search request to the search service.

[0309] The search request is used to request to obtain search results corresponding to the search information. The search request includes the search information.

[0310] Step 540: Receive a search request.

[0311] Step 541: In response to the search request, the search service obtains the search results corresponding to the search information, and sends the search results corresponding to the search information to the search management module.

[0312] Combined with the above, it can be seen that the search service stores the index data constructed by the search module in the gallery APP. Therefore, the search service can find the search results corresponding to the search information in the index data according to the search information. The search results are then sent to the search management module. Exemplarily, the search results corresponding to the search information can be sent to the search management module in the form of a list. The search results corresponding to the search information may include picture information and video information corresponding to the search information.

[0313] In some examples, the search results corresponding to the search information include the image name of the image to be detected, the image address of the image to be detected, the image identifier of the image to be detected, the name of the video, the identifier of the video, the address of the video, etc.

[0314] Step 542: The search management module receives search results corresponding to the search information.

[0315] Step 543: The search management module sends the search results corresponding to the search information to the APP to be processed.

[0316] Step 544: The APP to be processed receives the search results corresponding to the search information, and sends a search thumbnail acquisition request to the thumbnail management module.

[0317] The search thumbnail acquisition request is used to request to acquire at least one thumbnail corresponding to the search result. The search thumbnail acquisition request includes the search result.

[0318] When the APP to be processed obtains the search results corresponding to the search information, it can send a search thumbnail acquisition request to the thumbnail management module, so that the thumbnail management module generates at least one thumbnail corresponding to the search result based on the search results corresponding to the search information in the search thumbnail acquisition request.

[0319] In some examples, the thumbnail management module generates at least one thumbnail corresponding to the search results based on the search results corresponding to the search information in the search thumbnail acquisition request. Specifically, the thumbnail management module can generate a thumbnail corresponding to each image and a thumbnail corresponding to each video in sequence based on the image name of the image to be detected, the image address of the image to be detected, the image identifier of the image to be detected, the name of the video, the identifier of the video and the address of the video in the search results.

[0320] Step 545: The thumbnail management module receives a search thumbnail acquisition request.

[0321] Step 546: In response to the search thumbnail acquisition request, the thumbnail management module generates at least one thumbnail corresponding to the search result based on the search result in the search thumbnail acquisition request, and sends at least one thumbnail corresponding to the search result to the APP to be processed.

[0322] After the thumbnail management module receives the search thumbnail acquisition request, the thumbnail management module can sequentially generate at least one thumbnail corresponding to the search result based on the search result carried in the search thumbnail acquisition request. After generating at least one thumbnail corresponding to the search result, the at least one thumbnail corresponding to the search result can be sent to the APP to be processed, so that the APP to be processed displays at least one thumbnail corresponding to the search result to the user.

[0323] It can be understood that the process in which the thumbnail management module generates at least one thumbnail corresponding to the search results (i.e., a thumbnail corresponding to each image and a thumbnail corresponding to each video) in sequence based on the image name of the image to be detected, the image address of the image to be detected, the image identifier of the image to be detected, the name of the video, the identifier of the video and the address of the video in the search results is similar to the process in which the thumbnail management module in step 527 generates a thumbnail of each picture corresponding to the first portrait based on the picture information corresponding to the first portrait carried in the first portrait thumbnail acquisition request, and will not be repeated here.

[0324] In some examples, combined with step 527, it can be seen that before the thumbnail management module sends at least one thumbnail corresponding to the search result to the APP to be processed, the thumbnail management module also needs to determine whether the size of at least one thumbnail corresponding to the search result exceeds a preset threshold. In the case where the size of at least one thumbnail corresponding to the search result exceeds the preset threshold, it is also necessary to scale the at least one thumbnail corresponding to the search result so that the size of the at least one thumbnail corresponding to the scaled search result does not exceed the preset threshold. In the case where the size of at least one thumbnail corresponding to the search result does not exceed the preset threshold, the thumbnail management module can directly send at least one thumbnail corresponding to the search result to the APP to be processed.

[0325] Among them, the process of scaling at least one thumbnail corresponding to the search result can be: directly taking half of the size of at least one thumbnail corresponding to the search result to obtain at least one thumbnail corresponding to the scaled search result. Then continue to determine whether the size of at least one thumbnail corresponding to the scaled search result exceeds a preset threshold. If not, directly send at least one thumbnail corresponding to the scaled search result to the APP to be processed. If it exceeds, continue to take half of at least one thumbnail corresponding to the scaled search result until the size of at least one thumbnail corresponding to the final search result does not exceed the preset threshold.

[0326] Step 547: The APP to be processed receives at least one thumbnail corresponding to the search results, and displays at least one thumbnail corresponding to the search results.

[0327] After the APP to be processed receives at least one thumbnail corresponding to the search result, the APP to be processed will display at least one thumbnail corresponding to the search result. When the user views at least one thumbnail corresponding to the search result, the user can perform functional operations such as editing and sharing on one or more thumbnails of the at least one thumbnail corresponding to the search result based on needs. The functional operations that the user can perform are related to the functions that the APP to be processed can provide, and this disclosure does not limit this.

[0328] Exemplary, reference Fig. 9 The display interface of the APP to be processed shown in (a) includes a search button, and the user can click the search button based on the needs to enter the Fig.11 The search interface shown in (a) of FIG. 1 includes a search box 1101 and a search control 1102. A user may enter a search term "cat" in the search box 1101. In response to a click operation of the user on the search control 1102, Fig.11 (a) in the figure shows search prompt information 1103 and thumbnails 1104 corresponding to the search results. The search prompt information 1103 includes multiple search prompts, for example, the search prompts may be felines, cats, and giant pandas. The search prompt information is used to represent more detailed category information corresponding to the search terms, so that the user can quickly determine the required media data in the more detailed classification results.

[0329] After the APP to be processed displays the search prompt word information corresponding to the search word, the user can further select the search prompt word in the search prompt word information to quickly locate the required media data.

[0330] For example, in response to a user clicking on the search prompt word "giant panda" in the search prompt word information, the APP to be processed displays the following Fig.11 The interface shown in (b) is as follows: Fig.11(b) in the figure displays search hint word information 1105 corresponding to the search hint word “giant panda” and thumbnail 1106 corresponding to the search hint word “giant panda”.

[0331] Generally, the search term input by the user may not be accurate enough, so after receiving the search term input by the user, the accuracy of the search term can be determined first. If the accuracy of the search term meets the requirements, the search prompt term information and search results corresponding to the search term can be found based on the media data in the electronic device. If the accuracy of the search term does not meet the requirements, the standard search term corresponding to the search term can be determined based on the search term, and then the search prompt term information and search results corresponding to the standard search term can be found based on the media data in the electronic device.

[0332] Exemplarily, the process of determining the standard search term corresponding to the search term according to the search term may be: executing a fuzzy matching algorithm on the search term to match the standard search term corresponding to the search term. For example, for the search term "cat" input by the user, the standard search term "feline" corresponding to the search term "cat" may be determined by the fuzzy matching algorithm.

[0333] When the standard search term is a general term for a class of objects, the APP to be processed can obtain the associated words (also called subordinate category words) corresponding to the standard search term, and then. For example, the standard search term is "feline animals", "feline animals" can be understood as a general term for a class of objects, which can contain multiple subordinate category words, such as "cat" and "giant panda". Therefore, the APP to be processed can obtain the search results corresponding to the standard search term based on the associated words corresponding to the standard search term.

[0334] When the standard search term is a lower category word, the APP to be processed can further classify the standard search term from other dimensions to enable users to quickly locate the required media data based on the classification results. For example, if the standard search term is "giant panda", the standard search term can be further classified from dimensions such as location and year (for example, Beijing, Chongqing, 2022). Therefore, the APP to be processed can further classify the search results corresponding to the standard search term according to dimensions such as location and year, and display the classification results.

[0335] Based on the above scheme, it can be known that the present disclosure classifies and processes media data through the gallery APP, can obtain classification information corresponding to the media data, and update the classification information corresponding to the media data to the gallery database. Since the APP to be processed is set with multiple classification options, when the user uses the APP to be processed, if the user needs to find media data, the first option corresponding to the required media data can be filtered out from the multiple classification options, which greatly reduces the search range of the media data. Since multiple classification options are generated based on the classification information of the media data, the classification information of the media data is stored in the gallery database, the APP to be processed cannot directly access the gallery database, and the media data middle station APP will update the classification information of the media data in the gallery database in time, so when the user selects the first option from multiple classification options, the APP to be processed can obtain the thumbnail of the media data corresponding to the first option through the media data middle station APP. In this way, the search efficiency of media data can be improved, thereby improving user satisfaction.

[0336] For ease of understanding, the following Fig.12 The data processing method provided by the embodiment of the present disclosure is described. Fig.12 As shown, the data processing method may include the following steps 1201-1204.

[0337] Step 1201: Acquire media data.

[0338] Among them, media data and the above Figure 5 The multiple images to be detected in step 501 in the illustrated embodiment are similar.

[0339] Step 1202: The first application classifies the media data to obtain classification information corresponding to the media data, and updates the classification information corresponding to the media data to the gallery database.

[0340] Among them, the first application and the above Figure 5 The gallery APP in step 501 in the illustrated embodiment is similar.

[0341] The first application in the embodiment of the present disclosure performs classification processing on the media data to obtain classification information corresponding to the media data, and updates the classification information corresponding to the media data to the gallery database. Figure 5 Steps 501 to 508 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0342] In some examples, the first application classifies the media data to obtain classification information corresponding to the media data, including: the first application performs face detection on the media data to determine whether the photographed objects in the media data include faces; if the photographed objects in the media data do not include faces, the first application performs tag classification processing and location classification processing on the media data to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes category information corresponding to the media data and location information corresponding to the media data to be processed.

[0343] In the embodiment of the present disclosure, the first application performs face detection processing on the media data to determine whether the photographed object in the media data includes a face; when the photographed object in the media data does not include a face, the first application performs tag classification processing and location classification processing on the media data to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes category information corresponding to the media data and location information corresponding to the media data to be processed. Figure 5 Steps 501 to 505 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0344] In some examples, after determining whether the photographed subject in the media data includes a face, the method also includes: in the case that the photographed subject in the media data includes a face, a first application performs facial feature extraction processing and face clustering processing on the media data to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes portrait information corresponding to the media data; wherein the portrait information corresponding to the media data includes portrait album identification information corresponding to the media data, and the portrait album identification information corresponding to the media data is used to characterize the portrait data of the photographed subject in the media data.

[0345] In the embodiment of the present disclosure, the first application performs face detection processing on the media data to determine whether the photographed object in the media data includes a face; in the case where the photographed object in the media data includes a face, the first application performs face feature extraction processing and face clustering processing on the media data to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes portrait information corresponding to the media data; wherein the portrait information corresponding to the media data includes portrait album identification information corresponding to the media data, and the portrait album identification information corresponding to the media data is used to characterize the portrait data of the photographed object in the media data, and the above-mentioned Figure 5 Step 507 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0346] In some examples, after the first application classifies the media data and obtains classification information corresponding to the media data, the method also includes: the first application constructs index data based on the classification information corresponding to the media data, the index data including a correspondence between the media data and the classification information corresponding to the media data; the first application sends the index data to a search service.

[0347] The first application in the disclosed embodiment constructs index data based on the classification information corresponding to the media data, and the index data includes the corresponding relationship between the media data and the classification information corresponding to the media data; the first application sends the index data to the search service, which can refer to the above Figure 5 Step 510 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0348] In some examples, after the classification information corresponding to the media data is updated to the gallery database, the method also includes: the gallery database generates a transaction database based on the category information corresponding to the media data; generates a location database based on the location information corresponding to the media data; and generates a portrait database based on the portrait information corresponding to the media data; wherein the transaction database is used to characterize the correspondence between the media data and the category information corresponding to the media data; the location database is used to characterize the correspondence between the media data and the location information corresponding to the media data; and the portrait database is used to characterize the correspondence between the media data and the portrait information corresponding to the media data.

[0349] The gallery database in the disclosed embodiment generates a transaction database based on the category information corresponding to the media data; generates a location database based on the location information corresponding to the media data; and generates a portrait database based on the portrait information corresponding to the media data; wherein the transaction database is used to characterize the correspondence between the media data and the category information corresponding to the media data; the location database is used to characterize the correspondence between the media data and the location information corresponding to the media data; and the portrait database is used to characterize the correspondence between the media data and the portrait information corresponding to the media data. Figure 5 Step 506 and step 509 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0350] Step 1203: When the second application determines that the gallery database is updated, the second application obtains classification information corresponding to the media data from the gallery database.

[0351] Among them, the second application and the above Figure 5 The media data middleware APP in step 512 in the illustrated embodiment is similar.

[0352] In the embodiment of the present disclosure, when the second application determines that the gallery database is updated, the second application obtains the classification information corresponding to the media data from the gallery database, which can refer to the above Figure 5Step 512 and step 513 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0353] Step 1204: The third application receives a user's selection operation on a first option among multiple classification options, and sends a media data acquisition request to the second application, where the first option is any one of a portrait option, a location option, and a transaction option; the media data acquisition request is used to request a thumbnail of the media data corresponding to the first option; and the multiple classification options are generated based on the classification information of the media data.

[0354] Among them, the third application and the above Figure 5 The APPs to be processed in step 515 in the illustrated embodiment are similar.

[0355] In the embodiment of the present disclosure, the third application receives a user's selection operation of a first option among multiple classification options, and sends a media data acquisition request to the second application, wherein the first option is any one of a portrait option, a location option, and a business option; the media data acquisition request is used to request to obtain a thumbnail of the media data corresponding to the first option; the multiple classification options are generated based on the classification information of the media data, and can refer to the above Figure 5 Step 515, step 522 and step 529 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0356] Step 1205: The second application receives the media data acquisition request, and in response to the media data acquisition request, generates a thumbnail of the media data corresponding to the first option based on the classification information corresponding to the media data, and sends the thumbnail of the media data corresponding to the first option to the third application.

[0357] The second application in the disclosed embodiment receives the media data acquisition request, and in response to the media data acquisition request, generates a thumbnail of the media data corresponding to the first option based on the classification information corresponding to the media data, and sends the thumbnail of the media data corresponding to the first option to the third application. Figure 5 Steps 516 to 520, steps 522 to 527, and steps 529 to 534 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0358] In some examples, the first option includes a first sub-portrait option and a second sub-portrait option in the portrait option; the first sub-portrait option corresponds to the first portrait, and the second sub-portrait option corresponds to the second portrait.

[0359] Among them, the first sub-portrait option and the above Figure 5 The portrait 1 option in step 531 in the embodiment shown is similar. The second sub-portrait option is similar to the above Figure 5The portrait 2 option in step 531 in the embodiment shown is similar. Figure 5 The second portrait is similar to the portrait 1 in step 531 in the embodiment shown. Figure 5 The portrait 2 in step 531 in the illustrated embodiment is similar.

[0360] In response to a media data acquisition request, the second application generates a thumbnail of the media data corresponding to the first option based on the classification information corresponding to the media data, including: in response to the media data acquisition request, the second application determines the first album tag information corresponding to the first portrait based on the portrait album identification information corresponding to the media data; the first album tag information corresponds to at least one first media data; the photographed object in the first media data includes the first portrait; it is determined whether the portrait album identification information corresponding to each first media data in at least one first media data contains the second album tag information, and the media data containing the second album tag information in at least one first media data is determined as the second media data; the photographed object in the second media data includes at least the first portrait and the second portrait; the second application generates a thumbnail of the media data corresponding to the first option based on the second media data.

[0361] In response to a media data acquisition request in the disclosed embodiment, the second application determines first album tag information corresponding to a first portrait based on portrait album identification information corresponding to the media data; the first album tag information corresponds to at least one first media data; the photographed subject in the first media data includes a first portrait; it is determined whether the portrait album identification information corresponding to each first media data in at least one first media data contains second album tag information, and the media data containing the second album tag information in at least one first media data is determined as second media data; the photographed subject in the second media data includes at least the first portrait and the second portrait; the second application generates a thumbnail of the media data corresponding to the first option based on the second media data, which can be referred to above Figure 5 Steps 529 to 534 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0362] In some examples, sending a thumbnail of media data corresponding to a first option to a third application includes: determining whether the size of the thumbnail of the media data corresponding to the first option exceeds a preset threshold; if the size of the thumbnail of the media data corresponding to the first option exceeds the preset threshold, scaling the thumbnail of the media data corresponding to the first option to obtain a scaled thumbnail of the media data corresponding to the first option; if the size of the thumbnail of the media data corresponding to the first option after scaling does not exceed the preset threshold, sending the scaled thumbnail of the media data corresponding to the first option to the third application.

[0363] In the embodiment of the present disclosure, it is determined whether the size of the thumbnail of the media data corresponding to the first option exceeds a preset threshold; if the size of the thumbnail of the media data corresponding to the first option exceeds the preset threshold, the thumbnail of the media data corresponding to the first option is scaled to obtain a scaled thumbnail of the media data corresponding to the first option; if the size of the thumbnail of the media data corresponding to the first option after scaling does not exceed the preset threshold, sending the scaled thumbnail of the media data corresponding to the first option to the third application can refer to the above Figure 5 Step 527 and step 534 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0364] In some examples, the preset threshold is 500KB.

[0365] Step 1206: The third application receives and displays a thumbnail of the media data corresponding to the first option.

[0366] The third application in the embodiment of the present disclosure receives and displays the thumbnail of the media data corresponding to the first option. Figure 5 Step 521, step 528 and step 535 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0367] In some examples, the method also includes: the second application receives a media data search request from a third application, the media data search request includes a search term, and the media data search request is used to request to obtain media data corresponding to the search term; in response to the media data search request, the second application obtains the media data corresponding to the search term based on index data in the search service, and sends the media data corresponding to the search term to the third application; the third application receives and displays the media data corresponding to the search term.

[0368] In the disclosed embodiment, the second application receives a media data search request from a third application, the media data search request includes a search term, and the media data search request is used to request to obtain media data corresponding to the search term; in response to the media data search request, the second application obtains the media data corresponding to the search term based on the index data in the search service, and sends the media data corresponding to the search term to the third application; the third application receives and displays the media data corresponding to the search term, which can refer to the above Figure 5 Steps 536 to 547 in the illustrated embodiment will not be described in detail in the embodiment of the present disclosure.

[0369] For example, Fig.13 A schematic diagram of the structure of a media data search device 1300 is shown. Fig.13 As shown, the media data searching device 1300 may include: an acquisition module 1301, a processing module 1302, a sending module 1303, a receiving module 1304 and a display module 1305, etc.

[0370] The acquisition module 1301 is configured to acquire media data, and when the second application determines that the gallery database is updated, the second application acquires the classification information corresponding to the media data from the gallery database. The processing module 1302 is configured to classify the media data for the first application, obtain the classification information corresponding to the media data, and update the classification information corresponding to the media data to the gallery database. The sending module 1303 is configured to receive the user's selection operation on the first option among multiple classification options, and send a media data acquisition request to the second application, the first option is any one of the portrait option, the location option and the transaction option; the media data acquisition request is used to request to obtain a thumbnail of the media data corresponding to the first option; the multiple classification options are generated based on the classification information of the media data; and send the thumbnail of the media data corresponding to the first option to the third application. The receiving module 1304 is configured to receive the media data acquisition request for the second application, and generate a thumbnail of the media data corresponding to the first option based on the classification information corresponding to the media data in response to the media data acquisition request. The display module 1305 is configured to receive and display the thumbnail of the media data corresponding to the first option for the third application.

[0371] In one possible implementation, the processing module 1302 is further configured as a first application to perform face detection processing on the media data to determine whether the photographed object in the media data includes a face; when the photographed object in the media data does not include a face, the first application performs marking classification processing and location classification processing on the media data to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes category information corresponding to the media data and location information corresponding to the media data to be processed.

[0372] In one possible implementation, the processing module 1302 is further configured to, when the photographed subject in the media data includes a face, perform facial feature extraction processing and face clustering processing on the media data by the first application to obtain classification information corresponding to the media data, wherein the classification information corresponding to the media data includes portrait information corresponding to the media data; wherein the portrait information corresponding to the media data includes portrait album identification information corresponding to the media data, and the portrait album identification information corresponding to the media data is used to characterize the portrait data of the photographed subject in the media data.

[0373] In one possible implementation, the processing module 1302 is further configured for the first application to construct index data based on the classification information corresponding to the media data, and the index data includes a correspondence between the media data and the classification information corresponding to the media data; the sending module 1303 is further configured for the first application to send the index data to the search service.

[0374] In one possible implementation, the receiving module 1304 is further configured for the second application to receive a media data search request from a third application, where the media data search request includes a search term, and the media data search request is used to request to obtain media data corresponding to the search term; the processing module 1302 is further configured for the second application to obtain the media data corresponding to the search term based on the index data in the search service in response to the media data search request, and send the media data corresponding to the search term to the third application; the display module 1305 is further configured for the third application to receive and display the media data corresponding to the search term.

[0375] In one possible implementation, the first option includes a first sub-portrait option and a second sub-portrait option in a portrait option; the first sub-portrait option corresponds to a first portrait, and the second sub-portrait option corresponds to a second portrait; the processing module 1302 is also configured to respond to a media data acquisition request, and the second application determines the first album tag information corresponding to the first portrait based on the portrait album identification information corresponding to the media data; the first album tag information corresponds to at least one first media data; the photographed object in the first media data includes the first portrait; determine whether the portrait album identification information corresponding to each first media data in at least one first media data contains the second album tag information, and determine the media data containing the second album tag information in at least one first media data as the second media data; the photographed object in the second media data includes at least the first portrait and the second portrait; the second application generates a thumbnail of the media data corresponding to the first option based on the second media data.

[0376] In one possible implementation, the processing module 1302 is further configured to generate a transaction database from the gallery database according to the category information corresponding to the media data; generate a location database according to the location information corresponding to the media data; and generate a portrait database according to the portrait information corresponding to the media data; wherein the transaction database is used to characterize the correspondence between the media data and the category information corresponding to the media data; the location database is used to characterize the correspondence between the media data and the location information corresponding to the media data; and the portrait database is used to characterize the correspondence between the media data and the portrait information corresponding to the media data.

[0377] In a possible implementation, the processing module 1302 is further configured to determine whether the size of the thumbnail of the media data corresponding to the first option exceeds a preset threshold; if the size of the thumbnail of the media data corresponding to the first option exceeds the preset threshold, scale the thumbnail of the media data corresponding to the first option to obtain a scaled thumbnail of the media data corresponding to the first option. If the size of the thumbnail of the media data corresponding to the first option after scaling does not exceed the preset threshold, send the scaled thumbnail of the media data corresponding to the first option to the third application.

[0378] In a possible implementation, the preset threshold is 500 KB.

[0379] It should be understood that the division of units or modules (hereinafter referred to as units) in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or physically separated. And the units in the device can all be implemented in the form of software calling through processing elements; or all in the form of hardware; or some units can be implemented in the form of software calling through processing elements, and some units can be implemented in the form of hardware.

[0380] For example, each unit can be a separately established processing element, or it can be integrated in a certain chip of the device. In addition, it can also be stored in a memory in the form of a program, and called and executed by a certain processing element of the device. In addition, all or part of these units can be integrated together, or they can be implemented independently. The processing element here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each unit above can be implemented by an integrated logic circuit of hardware in the processor element or in the form of software called by a processing element.

[0381] It should be understood that each step in the above method embodiment provided by the present disclosure can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present disclosure can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0382] In one example, the unit in the above apparatus may be one or more integrated circuits configured to implement the above method, such as one or more ASICs, or one or more DSPs, or one or more FPGAs, or a combination of at least two of these integrated circuit forms.

[0383] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a CPU or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system on a chip SOC.

[0384] In one implementation, the units of the above apparatus implementing the corresponding steps in the above method can be implemented in the form of a processing element scheduling program. For example, the apparatus may include a processing element and a storage element, and the processing element calls the program stored in the storage element to execute the method of the above method embodiment. The storage element may be a storage element on the same chip as the processing element, that is, an on-chip storage element.

[0385] In another implementation, the program for executing the above method may be in a storage element on a different chip from the processing element, i.e., an off-chip storage element. In this case, the processing element calls or loads the program from the off-chip storage element to the on-chip storage element to call and execute the method of the above method embodiment.

[0386] For example, the embodiments of the present disclosure may also provide a device, such as an electronic device, which may include a processor and a memory for storing instructions executable by the processor. When the processor is configured to execute the above instructions, the electronic device implements the data processing method of the above embodiment. The memory may be located inside the electronic device or outside the electronic device. And the processor includes one or more.

[0387] In another implementation, the unit of the device implementing each step in the above method may be configured as one or more processing elements, which may be arranged on the corresponding electronic device, and the processing element here may be an integrated circuit, for example: one or more ASICs, or one or more DSPs, or one or more FPGAs, or a combination of these integrated circuits. These integrated circuits may be integrated together to form a chip.

[0388] For example, the present disclosure also provides a chip, which can be applied to the above electronic device. The chip includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected through lines; the processor receives and executes computer instructions from the memory of the electronic device through the interface circuit to implement the method in the above method embodiment.

[0389] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by an electronic device, the electronic device can implement the above-mentioned data processing method.

[0390] The disclosed embodiments also provide a computer program product, including computer instructions for the electronic device to run as described above, and when the computer instructions are run in the electronic device, the electronic device can implement the data processing method as described above. Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0391] In the several embodiments provided in the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0392] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0393] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0394] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, such as a program. The software product is stored in a program product, such as a computer-readable storage medium, including a number of instructions to enable a terminal device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the methods of each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, ROM, RAM, disk or optical disk and other media that can store program codes.

[0395] For example, the embodiments of the present disclosure may also provide a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by an electronic device, the electronic device implements the data processing method in the aforementioned method embodiment.

[0396] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present disclosure should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A media data search method, characterized in that: Applied to an electronic device, the electronic device includes a first application, a second application and a third application, and the method includes: Get media data; The first application performs classification processing on the media data to obtain classification information corresponding to the media data, and updates the classification information corresponding to the media data to a gallery database; When the second application determines that the gallery database is updated, the second application obtains classification information corresponding to the media data from the gallery database; Displaying a plurality of classification options in a display interface corresponding to the third application; the plurality of classification options are generated based on classification information of the media data; The third application receives a selection operation of a first option from among a plurality of classification options by a user, and sends a media data acquisition request to the second application, wherein the first option is any one of a portrait option, a location option, and a business option; the media data acquisition request is used to request acquisition of a thumbnail of media data corresponding to the first option; the first option includes a first sub-portrait option and a second sub-portrait option in the portrait option; the first sub-portrait option corresponds to a first portrait, and the second sub-portrait option corresponds to a second portrait; the second application receives the media data acquisition request, and in response to the media data acquisition request, generates a thumbnail of the media data corresponding to the first option based on the classification information corresponding to the media data, and sends the thumbnail of the media data corresponding to the first option to the third application; The third application receives and displays a thumbnail of the media data corresponding to the first option; The step of generating, in response to the media data acquisition request, a thumbnail of the media data corresponding to the first option based on the classification information corresponding to the media data, includes: In response to the media data acquisition request, the second application determines, based on the portrait album identification information corresponding to the media data, first album tag information corresponding to the first portrait; the first album tag information corresponds to at least one first media data; the photographed object in the first media data includes the first portrait; determining whether the portrait album identification information corresponding to each first media data in the at least one first media data includes the second album marking information, and determining the media data in the at least one first media data including the second album marking information as the second media data; the photographed objects in the second media data include at least the first portrait and the second portrait; The second application generates a thumbnail of the media data corresponding to the first option based on the second media data.

2. The method according to claim 1, characterized in that The first application classifies the media data to obtain classification information corresponding to the media data, including: The first application performs face detection processing on the media data to determine whether the photographed object in the media data includes a face; When the photographed object in the media data does not include a human face, the first application performs marking classification processing and location classification processing on the media data to obtain classification information corresponding to the media data, and the classification information corresponding to the media data includes category information corresponding to the media data and location information corresponding to the media data to be processed.

3. The method according to claim 2, characterized in that After determining whether the photographed object in the media data includes a face, the method further includes: In a case where the photographed object in the media data includes a face, the first application performs face feature extraction processing and face clustering processing on the media data to obtain classification information corresponding to the media data, wherein the classification information corresponding to the media data includes portrait information corresponding to the media data; The portrait information corresponding to the media data includes portrait album identification information corresponding to the media data, and the portrait album identification information corresponding to the media data is used to represent the portrait data of the photographed object in the media data.

4. The method according to any one of claims 1 to 3, characterized in that After the first application classifies the media data to obtain classification information corresponding to the media data, the method further includes: The first application constructs index data based on the classification information corresponding to the media data, the index data including a correspondence between the media data and the classification information corresponding to the media data; The first application sends the index data to a search service.

5. The method according to claim 4, characterized in that The method further comprises: The second application receives a media data search request from a third application, the media data search request including a search term, and the media data search request is used to request to obtain media data corresponding to the search term; In response to the media data search request, the second application obtains the media data corresponding to the search term based on the index data in the search service, and sends the media data corresponding to the search term to the third application; The third application receives and displays the media data corresponding to the search term.

6. The method according to claim 3, characterized in that After updating the classification information corresponding to the media data to the gallery database, the method further includes: The gallery database generates a transaction database according to the category information corresponding to the media data; generates a location database according to the location information corresponding to the media data; and generates a portrait database according to the portrait information corresponding to the media data; Among them, the transaction database is used to characterize the correspondence between the media data and the category information corresponding to the media data; the location database is used to characterize the correspondence between the media data and the location information corresponding to the media data; the portrait database is used to characterize the correspondence between the media data and the portrait information corresponding to the media data.

7. The method according to any one of claims 1 to 3, characterized in that The sending a thumbnail of the media data corresponding to the first option to the third application includes: Determining whether a size of a thumbnail of the media data corresponding to the first option exceeds a preset threshold; When the size of the thumbnail of the media data corresponding to the first option exceeds a preset threshold, scaling the thumbnail of the media data corresponding to the first option to obtain a scaled thumbnail of the media data corresponding to the first option; When the size of the thumbnail of the media data corresponding to the scaled first option does not exceed a preset threshold, the thumbnail of the media data corresponding to the scaled first option is sent to the third application.

8. The method according to claim 7, characterized in that The preset threshold is 500KB.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory for storing instructions executable by the processor; when the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having computer program instructions stored thereon; characterized in that: When the computer program instructions are executed by an electronic device, the electronic device implements the method according to any one of claims 1 to 8.

Images