Expression management method and device, medium, electronic equipment and program product

Abstract

The invention provides an expression management method and device, a medium, electronic equipment and a program product. The method comprises the following steps: grouping a plurality of expressions in a local expression library of an application program, determining a classification label of each group according to a grouping result, for each expression in each group, obtaining a category to which an object with the maximum pixel proportion in the expressions belongs, and determining a subject category label of the expressions; extracting character information in the expression, and determining a character label according to the character information; and determining the classification label, the subject category label and the text label as indexes of the expression, and through the management mode, searching the corresponding expression according to the indexes, thereby improving the efficiency of searching the expression.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an expression management method, device, medium, electronic device, and program product. Background Technology

[0002] Emoticons in applications can be seen as an important supplement to online text communication, conveying a user's status or emotions and increasing the fun of online communication. However, current applications lack emoticon management functions. Faced with a large number of emoticons added locally by users, sending a specific emoticon requires users to manually search through all the added emoticons, affecting communication efficiency. Summary of the Invention

[0003] In view of this, the present disclosure provides an expression management method, apparatus, medium, electronic device, and program product.

[0004] According to a first aspect of the present disclosure, an emoticon management method is provided, applied to a client application. The method includes: grouping multiple emoticons in the application's local emoticon library; determining a category label for each group based on the grouping result; for each emoticon in each group, obtaining the category to which the object with the largest pixel proportion in the emoticon belongs, and determining a main category label for the emoticon; extracting text information from the emoticon and determining a text label based on the text information; and determining the category label, the main category label, and the text label as an index for the emoticon.

[0005] In some embodiments, the method further includes: in response to obtaining a keyword input by a user, searching for the keyword in the index of each emoji in the local emoji library; displaying search results matching the keyword; and outputting an emoji based on the user's selection in the search results.

[0006] In some embodiments, obtaining the category of the object with the largest pixel proportion in the expression includes: using a pre-trained fully convolutional neural network to obtain the pixel proportion of each preset object in the image where the expression is located; and obtaining the category of the object with the largest pixel proportion from the pixel proportions occupied by all preset objects.

[0007] In some embodiments, grouping multiple emoticons in the application's local emoticon library includes: obtaining feature vectors of multiple emoticons in the application's local emoticon library; clustering based on the density between the multiple feature vectors to obtain a clustering result; and grouping the multiple emoticons in the local emoticon library based on the clustering result.

[0008] In some embodiments, the method further includes: in response to receiving an instruction to display expressions, displaying expressions from the local expression library in groups.

[0009] In some embodiments, the method further includes: in response to receiving an added emoji to be processed, determining the group to which the emoji to be processed belongs; obtaining the category tag, subject category tag, and text tag of the emoji to be processed; storing the emoji to be processed in the corresponding group, and adding an index determined based on the category tag, subject category tag, and text tag.

[0010] According to a second aspect of the present disclosure, an expression management device is provided, applied to a client application, the device comprising:

[0011] A grouping unit is used to group multiple emojis in the application's local emoji library and determine the classification label for each group based on the grouping results.

[0012] The acquisition unit is used to acquire, for each expression in each group, the category to which the object with the largest pixel proportion in the expression belongs, determine the main category label of the expression, extract the text information in the expression, and determine the text label based on the text information;

[0013] A determining unit is used to determine the category label, the main category label, and the text label as the index of the expression.

[0014] In some embodiments, the apparatus further includes a search unit for searching for keywords in the indexes of each emoji in the local emoji library in response to obtaining keywords input by the user; displaying search results matching the keywords; and outputting emojis based on the user's selection in the search results.

[0015] In some embodiments, the acquisition unit is specifically used to: use a pre-trained fully convolutional neural network to acquire the pixel ratio occupied by each preset object in the image where the expression is located; and acquire the category to which the object with the largest pixel ratio belongs from the pixel ratio occupied by all preset objects.

[0016] In some embodiments, the grouping unit is specifically used to: obtain feature vectors of multiple emoticons in the local emoticon library of the application; perform clustering based on the density between the multiple feature vectors to obtain clustering results; and group the multiple emoticons in the local emoticon library according to the clustering results.

[0017] In some embodiments, the apparatus further includes a processing unit, configured to, in response to receiving an added emoji to be processed, determine the group to which the emoji to be processed belongs; obtain the category tag, subject category tag, and text tag of the emoji to be processed; store the emoji to be processed in the corresponding group, and add an index determined based on the category tag, subject category tag, and text tag.

[0018] According to a third aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the methods described in the first aspect above.

[0019] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising:

[0020] processor;

[0021] Memory used to store processor-executable instructions;

[0022] The processor is configured as follows:

[0023] Multiple emoticons in the application's local emoticon library are grouped, and classification labels for each group are determined based on the grouping results;

[0024] For each expression in each group, obtain the category of the object with the largest pixel proportion in the expression, and determine the main category label of the expression;

[0025] Extract the text information from the emoticon and determine the text tag based on the text information;

[0026] The category label, the main category label, and the text label are determined as the index of the emoticon.

[0027] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program and instructions that, when executed by a processor, implement the steps of any of the methods described in the first aspect above.

[0028] The technical solutions provided in this disclosure may have the following beneficial effects:

[0029] Multiple emojis in the application's local emoji library are grouped, and category labels are determined for each group based on the grouping results. For each emoji in each group, the category to which the object with the largest pixel proportion in the emoji belongs is obtained, and the main category label of the emoji is determined. The text information in the emoji is extracted, and text labels are determined based on the text information. The category label, the main category label, and the text label are used as the index of the emoji. Through the above management method, the corresponding emoji can be searched according to the index, thereby improving the efficiency of emoji search.

[0030] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0031] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0032] Figure 1 A flowchart illustrating an exemplary embodiment of this disclosure of an expression management method;

[0033] Figure 2 This is a schematic diagram illustrating an embodiment of obtaining a subject category label according to an exemplary embodiment of the present disclosure;

[0034] Figure 3 This is a schematic diagram of the structure of an expression management device according to an exemplary embodiment of the present disclosure;

[0035] Figure 4 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0036] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0037] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0038] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0039] When communicating online through applications, emoticons can serve as an important supplement to online text communication. Compared to text, emoticons can not only convey a user's emotions more accurately, but also save users time in organizing their thoughts. For example, when a user wants to express happiness, they don't need to think about which words are more precise; they can simply send an emoticon that represents happiness.

[0040] During chats, users typically add their favorite emojis to their local emoji library, which grows over time. However, there's currently no management feature for this local emoji library; users have to manually search for specific emojis one by one, which is inefficient.

[0041] Therefore, this disclosure provides an emoticon management method that can be applied to client applications, such as chat applications. This disclosure enables effective management of local emoticons, reducing the time users spend searching for emoticons and improving communication efficiency.

[0042] The facial expression management method provided in this disclosure will now be described in detail with reference to the accompanying drawings.

[0043] Figure 1 A flowchart of an exemplary embodiment of this disclosure provides an expression management method, such as... Figure 1 As shown, the method includes steps 101 to 104.

[0044] In step 101, multiple emoticons in the application's local emoticon library are grouped, and the classification labels for each group are determined based on the grouping results.

[0045] A local emoji library refers to a database storing emojis added by users. These emojis can include those added from the internet, emoji collections added from the internet, and user-defined emojis. In this embodiment, multiple emojis added by the user from the internet can be grouped, with the grouping criteria determined by category tags for each group. The grouping criteria can be determined based on actual circumstances. For example, they can be grouped according to the meaning represented by the emojis; for instance, they can be grouped according to emotions such as happiness, anger, and depression. They can also be grouped according to the object category in the emojis; for instance, they can be grouped according to people, cats, dogs, and landscapes.

[0046] In step 102, for each expression in each group, the category of the object with the largest pixel proportion in the expression is obtained, and the main category label of the expression is determined.

[0047] It should be noted that the category to which the object with the largest pixel proportion in an emoji belongs is not necessarily the same as the category label of the emoji.

[0048] In step 103, the text information in the emoticon is extracted, and the text label is determined based on the text information.

[0049] For emoticons containing text, adding a text tag to the emoticon allows for quick location of the emoticon. For example, if an emoticon includes the phrase "You can achieve great things," then "You can achieve great things" can be identified as the text tag for that emoticon.

[0050] In this embodiment, text information can be extracted from the emoticons, and text tags can be determined based on keywords or semantic information in the text information. For example, for the emoticon that includes "You can achieve great things," "great things" is determined as the text tag for that emoticon.

[0051] In step 104, the category label, the main category label, and the text label are determined as the index of the expression.

[0052] This embodiment groups multiple emoticons in the application's local emoticon library, determines the category label for each group based on the grouping results, and for each emoticon in each group, obtains the category of the object with the largest pixel proportion in the emoticon to determine the main category label of the emoticon; extracts the text information in the emoticon, and determines the text label based on the text information; the category label, the main category label, and the text label are determined as the index of the emoticon. Through the above management method, the corresponding emoticon can be searched according to the index, thereby improving the efficiency of emoticon search.

[0053] When there is no text in the emoticon, the text tag position is empty; when the emoticon contains only text, the main category tag is "text," and the text tag is the specific text content in the emoticon. This disclosure, by defining category tags, main category tags, and text tags as the index for emoticons, enables a more comprehensive search of emoticons and improves the hit rate.

[0054] When indexing emoticons in a local emoticon library, in response to obtaining a keyword input by the user, the keyword is searched in the index of each emoticon in the local emoticon library; search results matching the keyword are displayed, and emoticons are output based on the user's selection in the search results.

[0055] The keywords entered by the user can be keywords extracted from the information entered by the user.

[0056] In one implementation, upon receiving a keyword input by the user, the user can sequentially search for the keyword in the category tags, subject category tags, and text tags of each emoji in the local emoji library. If any tag matching the keyword is found, the search result matching the keyword is displayed, that is, the emoji corresponding to the tag matching the keyword is displayed.

[0057] In another implementation, when a keyword input by the user is obtained, the keyword can be searched for in the category tags of each emoji in the local emoji library. If a category tag matching the keyword is found, the main category tag matching the keyword can be searched in the same category tag index, and / or the text tag matching the keyword can be searched in the same category tag index. The emoji corresponding to the main category tag and / or text tag matching the keyword can then be displayed.

[0058] In another implementation, upon receiving a keyword input by the user, the system first searches for the keyword in the text tags of each emoji in the local emoji library. If a text tag matching the keyword is found, the corresponding emoji is displayed; otherwise, the search proceeds to the main category tags of each emoji in the local emoji library. If a main category tag matching the keyword is found, the corresponding emoji is displayed; otherwise, the search continues in the category tags of each emoji in the local emoji library. If a category tag matching the keyword is found, the corresponding emoji is displayed; otherwise, no emoji is displayed.

[0059] For commonly used applications such as Xiaomi Office, WeChat, QQ, and Lark, after users add emojis to their local devices, these applications lack categorization and management of these emojis, and do not provide a local search function. Therefore, when users need to send a specific emoji during a chat, they can only manually search for it one by one, reducing communication efficiency and resulting in a poor user experience. Therefore, this disclosure provides a local emoji management method. When users add emojis locally, they can automatically categorize and group them. For each group of emojis, key information is extracted, including the name of the most frequently used object after image recognition, and the text extracted from the emoji. These are combined to form an emoji index. Based on this, when a user needs to use a specific emoji during a chat, they can quickly locate and send it by entering relevant keywords.

[0060] In some embodiments, grouping multiple emoticons in the application's local emoticon library may include: obtaining feature vectors of multiple emoticons in the application's local emoticon library; clustering based on the density between the multiple feature vectors to obtain a clustering result; and grouping the multiple emoticons in the local emoticon library based on the clustering result.

[0061] For users who already have locally stored emoji data added or saved from the internet, feature vectors are extracted from the local emoji library. For example, the Inception v3 model can be used to extract features from emojis, generating a 2048-dimensional feature vector. During implementation, each emoji can be pre-labeled with a numerical tag so that corresponding clusters can be achieved after the feature vectors are clustered.

[0062] Having obtained feature vectors for multiple facial expressions, feature clustering can be performed using a clustering algorithm to obtain clustering results. In one implementation, density-based spatial clustering of applications with noise (DBSCAN) can be used for clustering. Those skilled in the art will understand that, in addition to the above-mentioned clustering algorithms, other existing clustering algorithms can also be used, and this embodiment does not limit them.

[0063] After obtaining the clustering results, expressions of the same category are grouped into the same group to complete the expression grouping.

[0064] For each expression in each group after grouping, obtaining the category of the object with the largest pixel proportion in the expression can include: using a pre-trained fully convolutional neural network to obtain the pixel proportion of each preset object in the image where the expression is located; and obtaining the category of the object with the largest pixel proportion from the pixel proportions of all preset objects.

[0065] For example, the semantic segmentation FCN model can be used to identify different objects in an image containing an expression. See [link to object segmentation process] for details. Figure 2 The convolutional layers take facial expression images as input and reduce the dimensionality of the data through pooling layers. During training, a fully convolutional neural network model (FCN-8s) is trained using the ADE-20K (ADE20K Scene Parsing Challenge) dataset. The trained FCN-8s can segment an image of any size into 151 object categories and obtain the pixel proportion of each object, extracting the object category with the largest proportion in each facial expression image as the main category label.

[0066] In this embodiment of the disclosure, text in each facial expression can be extracted based on Optical Character Recognition (OCR) technology. OCR recognition refers to the recognition of optical characters through image processing and pattern recognition technologies. For example, text in facial expressions can be extracted based on a CRNN recognition model in end-to-end OCR technology. The CRNN network structure consists of three parts, from bottom to top:

[0067] 1. Convolutional layer, using a deep CNN, extracts features from the input image.

[0068] 2. Recurrent layer: Use bidirectional RNN (BLSTM) to predict the feature sequence and output the distribution of predicted labels (true values).

[0069] 3. Transcription layer: Using CTC loss, a series of tag distributions obtained from the recurrent layer are transformed into the final tag sequence.

[0070] The recognition process using CRNN can include: input image, image preprocessing, text region detection, feature extraction, and text recognition.

[0071] When emoticons in the local emoticon library are grouped, the emoticons in the local emoticon library can be displayed in groups upon receiving an instruction to display emoticons. Displaying emoticons from the local emoticon library in groups on the user interface improves the efficiency of manually searching for specific emoticons compared to displaying emoticons without grouping.

[0072] In this embodiment, an index can be built based on the Elasticsearch search engine. Elasticsearch is a distributed search and analytics engine that can be used for full-text search, structured search, and analysis, and can combine these three. In this embodiment, category tags, subject category tags, and text tags are combined to build an ElasticSearch index for users to perform fuzzy searches. That is, when searching, the keywords entered by the user can be object names, such as panda, dog, etc., or they can enter one or more characters contained in an emoji for location.

[0073] The above embodiments describe the management operations performed on existing emoticons in the local emoticon library. For newly added emoticons to the local emoticon library, the method may further include: in response to receiving an added emoticon to be processed, determining the group to which the emoticon to be processed belongs; obtaining the category tag, subject category tag, and text tag of the emoticon to be processed; storing the emoticon to be processed in the corresponding group, and adding an index determined based on the category tag, subject category tag, and text tag.

[0074] In other words, after grouping and managing the emoticons in the local emoticon library, for the added emoticon to be processed, the group to which the emoticon belongs is determined, and after determining the category tag, main category tag and text tag of the emoticon to be processed, the emoticon to be processed is stored in the corresponding group, and a corresponding index is added for the emoticon to be processed.

[0075] This disclosure allows for unified management of user-added emoticons and provides an efficient search tool, solving the technical problem of not being able to quickly locate the required emoticon and improving the user experience.

[0076] As can be seen from the above introduction, the technologies involved in this disclosure can be deployed in a lightweight manner, and the models involved can be quickly iterated and optimized according to needs. In addition, machine learning methods are used to control and process each link in the implementation process, which can reduce manual intervention and operation and maintenance costs.

[0077] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should know that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps may be performed in other orders or simultaneously.

[0078] Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by this disclosure.

[0079] Corresponding to the aforementioned application function implementation method embodiments, this disclosure also provides embodiments of application function implementation apparatus and corresponding terminals.

[0080] Figure 3 This is a schematic diagram of the structure of an expression management device according to an exemplary embodiment of the present disclosure, such as... Figure 3 As shown, the expression management device may include:

[0081] Grouping unit 301 is used to group multiple emoticons in the local emoticon library of the application and determine the classification label of each group based on the grouping results;

[0082] The acquisition unit 302 is used to acquire, for each expression in each group, the category to which the object with the largest pixel proportion in the expression belongs, determine the main category label of the expression, extract the text information in the expression, and determine the text label based on the text information;

[0083] The determining unit 303 is used to determine the classification label, the main category label, and the text label as the index of the expression.

[0084] This disclosure helps users quickly find the emojis they need, avoiding the inconvenience of manually searching through them one by one, improving communication efficiency and enhancing the user experience.

[0085] In some embodiments, the apparatus further includes a search unit for searching for keywords in the indexes of each emoji in the local emoji library in response to obtaining keywords input by the user; displaying search results matching the keywords; and outputting emojis based on the user's selection in the search results.

[0086] In some embodiments, the acquisition unit is specifically used to: use a pre-trained fully convolutional neural network to acquire the pixel ratio occupied by each preset object in the image where the expression is located; and acquire the category to which the object with the largest pixel ratio belongs from the pixel ratio occupied by all preset objects.

[0087] In some embodiments, the grouping unit is specifically used to: obtain feature vectors of multiple emoticons in the local emoticon library of the application; perform clustering based on the density between the multiple feature vectors to obtain clustering results; and group the multiple emoticons in the local emoticon library according to the clustering results.

[0088] In some embodiments, the apparatus further includes a processing unit, configured to, in response to receiving an added emoji to be processed, determine the group to which the emoji to be processed belongs; obtain the category tag, subject category tag, and text tag of the emoji to be processed; store the emoji to be processed in the corresponding group, and add an index determined based on the category tag, subject category tag, and text tag.

[0089] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0090] Accordingly, this disclosure provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to:

[0091] Multiple emoticons in the application's local emoticon library are grouped, and classification labels for each group are determined based on the grouping results;

[0092] For each expression in each group, obtain the category of the object with the largest pixel proportion in the expression, and determine the main category label of the expression;

[0093] Extract the text information from the emoticon and determine the text tag based on the text information;

[0094] The category label, the main category label, and the text label are determined as the index of the emoticon.

[0095] Figure 4 This is a schematic diagram illustrating the structure of an electronic device 400 according to an exemplary embodiment. For example, the electronic device 400 may be a user device, specifically a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, wearable device such as smartwatch, smart glasses, smart bracelet, smart running shoes, etc.

[0096] Reference Figure 4 The electronic device 400 may include one or more of the following components: processing component 402, memory 404, power supply component 406, multimedia component 408, audio component 410, input / output (I / O) interface 412, sensor component 414, and communication component 416.

[0097] Processing component 402 typically controls the overall operation of electronic device 400, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 402 may include one or more modules to facilitate interaction between processing component 402 and other components. For example, processing component 402 may include a multimedia module to facilitate interaction between multimedia component 408 and processing component 402.

[0098] Memory 404 is configured to store various types of data to support the operation of device 400. Examples of this data include instructions for any application or method operating on electronic device 400, contact data, phonebook data, messages, pictures, videos, etc. Memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0099] Power supply component 406 provides power to various components of electronic device 400. Power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 400.

[0100] Multimedia component 408 includes a screen that provides an output interface between the aforementioned electronic device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 408 includes a front-facing camera and / or a rear-facing camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0101] Audio component 410 is configured to output and / or input audio signals. For example, audio component 410 includes a microphone (MIC) configured to receive external audio signals when electronic device 400 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 404 or transmitted via communication component 416. In some embodiments, audio component 410 also includes a speaker for outputting audio signals.

[0102] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0103] Sensor assembly 414 includes one or more sensors for providing state assessments of various aspects of electronic device 400. For example, sensor assembly 414 can detect the on / off state of electronic device 400, the relative positioning of components such as the display and keypad of electronic device 400, changes in position of electronic device 400 or a component of electronic device 400, the presence or absence of user contact with electronic device 400, orientation or acceleration / deceleration of electronic device 400, and temperature changes of electronic device 400. Sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 414 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 414 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0104] Communication component 416 is configured to facilitate wired or wireless communication between electronic device 400 and other devices. Electronic device 400 can access wireless networks based on communication standards, such as WiFi, 4G or 5G, 4G LTE, 5G NR, or combinations thereof. In one exemplary embodiment, communication component 416 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the aforementioned communication component 416 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0105] In an exemplary embodiment, the electronic device 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0106] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as a memory 404 including instructions, which, when executed by a processor 420 of an electronic device 400, enables the electronic device 400 to perform an expression management method, the method including:

[0107] Multiple emoticons in the application's local emoticon library are grouped, and classification labels for each group are determined based on the grouping results;

[0108] For each expression in each group, obtain the category of the object with the largest pixel proportion in the expression, and determine the main category label of the expression;

[0109] Extract the text information from the emoticon and determine the text tag based on the text information;

[0110] The category label, the main category label, and the text label are determined as the index of the emoticon.

[0111] The non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0112] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0113] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An expression management method, characterized in that, The method, applied to a client application, includes: Multiple emoticons in the application's local emoticon library are grouped, and classification labels for each group are determined based on the grouping results; For each expression in each group, obtain the category of the object with the largest pixel proportion in the expression, and determine the main category label of the expression; Extract the text information from the emoticon and determine the text tag based on the text information; The category label, the main category label, and the text label are determined as the index of the emoticon.

2. The method according to claim 1, characterized in that, The method further includes: In response to receiving a keyword input by the user, the keyword is searched in the index of each emoji in the local emoji library; Display search results that match the keywords; The emoji is output based on the user's selection in the search results.

3. The method according to claim 1, characterized in that, Obtain the category of the object with the largest pixel proportion in the expression, including: Using a pre-trained fully convolutional neural network, the pixel ratio of each preset object in the image containing the expression is obtained; From the pixel proportions of all preset objects, obtain the category to which the object with the largest pixel proportion belongs.

4. The method according to claim 1, characterized in that, The grouping of multiple emojis from the application's local emoji library includes: Obtain the feature vectors of multiple emoticons from the application's local emoticon library; Clustering is performed based on the density among multiple feature vectors to obtain the clustering results; Based on the clustering results, multiple emoticons in the local emoticon library are grouped.

5. The method according to claim 1, characterized in that, The method further includes: In response to receiving an instruction to display emoticons, the emoticons in the local emoticon library are displayed in groups.

6. The method according to claim 1, characterized in that, The method further includes: In response to receiving an added emoji to be processed, determine the group to which the emoji to be processed belongs; Obtain the category label, main category label, and text label of the expression to be processed; The emoticons to be processed are stored in the corresponding groups, and indexes are added based on category tags, subject category tags, and text tags.

7. An expression management device, characterized in that, The device is used in a client application and includes: A grouping unit is used to group multiple emojis in the application's local emoji library and determine the classification label for each group based on the grouping results. The acquisition unit is used to acquire, for each expression in each group, the category to which the object with the largest pixel proportion in the expression belongs, determine the main category label of the expression, extract the text information in the expression, and determine the text label based on the text information; A determining unit is used to determine the category label, the main category label, and the text label as the index of the expression.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method described in any one of claims 1 to 6.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program and instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images