Internet community-based expression package recognition method and device, and storage medium

Abstract

The application relates to an Internet community-based emoticon package identification method and device and a storage medium. The method comprises the following steps: acquiring an image in an Internet community, and using a pre-trained image classification model to classify the image to determine whether the image belongs to an emoticon package category; when the image does not belong to the emoticon package category, using an OCR model to acquire a text detection result in the image; determining whether the text detection result contains a keyword in a preset keyword library; when the text detection result contains the keyword, determining whether the text detection result meets a preset filtering condition; when the text detection result meets the preset filtering condition, determining that the image belongs to the emoticon package category, and adding the image to an emoticon package recommendation pool. Through the technical scheme, the accuracy of emoticon package identification is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to an expression package recognition method and device based on an Internet community and a storage medium. BACKGROUND

[0002] As a new expression form, expression packages have been loved and used by more and more users in recent years. The humorous style of expression packages has a positive effect on the liveliness of a community atmosphere and the reduction of a user post threshold. Therefore, discovering and recommending expression packages in a community, especially popular expression packages, to users has a positive effect on the liveliness of a community.

[0003] On the other hand, a community has a high requirement for the precision and recall of expression package recognition. The materials of expression packages are mostly derived from stars, net reds, animations, cute pets, and cute children. If the recall of an expression package recognition model is not enough, this part of pictures will be judged as stars, net reds, animations, cute pets, and cute children. However, these expression package pictures are essentially emotional expressions of users, and therefore have a negative impact on a recommendation system. If some ordinary pictures that are not expression packages are recommended to users as expression packages, the purpose of reducing the user post threshold and enriching the emotional expression level cannot be achieved, and the community atmosphere and liveliness are negatively affected. SUMMARY

[0004] To overcome the problems in the related art, the present application provides an expression package recognition method and device based on an Internet community and a storage medium, thereby improving the accuracy of expression package recognition.

[0005] According to a first aspect of an embodiment of the present application, an expression package recognition method based on an Internet community is provided, and the method comprises the following steps.

[0006] An image in an Internet community is acquired, and a pre-trained image classification model is used to classify the image to determine whether the image belongs to an expression package category.

[0007] When the image does not belong to the expression package category, an OCR model is used to acquire a text detection result in the image.

[0008] It is determined whether the text detection result contains a keyword in a preset keyword library.

[0009] When the text detection result contains the keyword, it is determined whether the text detection result meets a preset filtering condition.

[0010] When the text detection result meets the preset filtering condition, it is determined that the image belongs to the expression package category, and the image is added to an expression package recommendation pool.

[0011] In an embodiment, preferably, the preset filtering condition comprises:

[0012] The effective matching proportion of the text detection result is greater than or equal to a first preset threshold value;

[0013] The IOU of the text box of the text detection result and the preset user ID watermark box is less than a second preset threshold value.

[0014] In an embodiment, preferably, the effective matching proportion is calculated by using the following first calculation formula:

[0015] Effective matching proportion = maximum matching text length / total length of text in text detection result;

[0016] The IOU of the text box of the text detection result and the preset user ID watermark box is calculated by using the following second calculation formula:

[0017] IOU = intersection area of text box and preset user ID watermark box / union area of text box and preset user ID watermark box.

[0018] In an embodiment, preferably, the method further comprises:

[0019] Obtaining an image MD5 feature library;

[0020] Determining whether a target image matching the image can be found in the image MD5 feature library;

[0021] When the target image is not found, monitoring whether the image is sent again within a first preset time period;

[0022] When the image is sent again within the first preset time period, recording the time t1 of the image being sent again;

[0023] Monitoring the first number of times the image is sent within a second preset time period after the time t1 of being sent again;

[0024] When the first number of times is greater than or equal to a first preset number of times, recommending in the post comment emoji package recommendation, and recording the second number of times the image is sent within t1 + second preset time period to t1 + third preset time period;

[0025] When the second number of times is greater than or equal to the first preset number of times, adding the image to the emoji package recommendation pool.

[0026] In an embodiment, preferably, the method further comprises:

[0027] Obtain image materials and historical image comment data in a historical meme topic, and perform meme category labeling to obtain a labeled training set;

[0028] Perform training using the training set and an EfficientNet model to obtain the pre-trained image classification model.

[0029] In an embodiment, preferably, the method further comprises:

[0030] When the image belongs to the meme category, the image is added to a meme recommendation pool.

[0031] According to a second aspect of an embodiment of the present application, an Internet community-based meme recognition device is provided, and the device comprises:

[0032] A classification module is configured to obtain images in an Internet community, and perform classification on the images using a pre-trained image classification model to determine whether the images belong to a meme category.

[0033] A recognition module is configured to, when the images do not belong to the meme category, obtain a text detection result in the images by using an OCR model.

[0034] A first determination module is configured to determine whether the text detection result contains a keyword in a preset keyword library.

[0035] A second determination module is configured to, when the text detection result contains the keyword, determine whether the text detection result satisfies a preset filtering condition.

[0036] A third determination module is configured to, when the text detection result satisfies the preset filtering condition, determine that the image belongs to the meme category, and add the image to a meme recommendation pool.

[0037] In an embodiment, preferably, the preset filtering condition comprises:

[0038] An effective matching proportion of the text detection result is greater than or equal to a first preset threshold value;

[0039] An IOU of a text box of the text detection result and a preset user ID watermark box is less than a second preset threshold value.

[0040] In an embodiment, preferably, the effective matching proportion is calculated using the following first calculation formula:

[0041] Effective matching proportion = maximum matching text length / total text length in the text detection result.

[0042] An IOU of a text box of the text detection result and a preset user ID watermark box is calculated by using the following second calculation formula:

[0043] IOU = intersection area of the text box and the preset user ID watermark box / union area of the text box and the preset user ID watermark box.

[0044] In an embodiment, preferably, the device further comprises:

[0045] A first acquisition module is configured to acquire an image MD5 feature library.

[0046] A fourth determination module is configured to determine whether a target image matching the image can be found in the image MD5 feature library.

[0047] A first monitoring module is configured to monitor whether the image is sent again within a first preset time period when the target image is not found.

[0048] A first recording module is configured to record a time t1 when the image is sent again within the first preset time period.

[0049] A second monitoring module is configured to monitor a first number of times the image is sent within a second preset time period after the time t1 when the image is sent again.

[0050] A second recording module is configured to recommend the image in a post comment emoji pack recommendation when the first number of times is greater than or equal to a first preset number of times, and record a second number of times the image is sent within a third preset time period after the time t1+ the second preset time period.

[0051] A processing module is configured to add the image to an emoji pack recommendation pool when the second number of times is greater than or equal to the first preset number of times.

[0052] In an embodiment, preferably, the device further comprises:

[0053] A second acquisition module is configured to acquire image materials in a historical emoji pack topic and historical image comment data, and perform emoji pack category labeling to obtain a labeled training set.

[0054] A training module is configured to train the training set and an EfficientNet model to obtain the pre-trained image classification model.

[0055] In an embodiment, preferably, the processing module is further configured to:

[0056] When the image belongs to an emoji pack category, the image is added to an emoji pack recommendation pool.

[0057] According to a third aspect of the embodiments of the present application, an Internet community-based meme recognition device is provided, the device comprising:

[0058] a processor;

[0059] a memory for storing processor-executable instructions;

[0060] wherein the processor is configured to:

[0061] obtain an image in an Internet community and classify the image using a pre-trained image classification model to determine whether the image belongs to a meme category;

[0062] when the image does not belong to the meme category, obtain a text detection result in the image through an OCR model;

[0063] determine whether the text detection result contains a keyword in a preset keyword library;

[0064] when the text detection result contains the keyword, determine whether the text detection result meets a preset filtering condition;

[0065] when the text detection result meets the preset filtering condition, determine that the image belongs to the meme category, and add the image to a meme recommendation pool.

[0066] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores computer instructions, and the instructions are executed by a processor to implement the steps of the method according to any one of the embodiments of the second aspect.

[0067] The technical solutions provided by the embodiments of the present application can include the following beneficial effects:

[0068] In the embodiments of the present application, the text box is filtered, which can reduce the false injury problem caused by sensitive word matching. Meanwhile, the present application recalls the meme that does not belong to the general type and does not contain text through the preset filtering condition.

[0069] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0070] The accompanying drawings, which are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the present application.

[0071] Figure 1 is a flowchart of an Internet community-based meme recognition method according to an exemplary embodiment.

[0072] Figure 2 This is a flowchart illustrating another method for recognizing emojis based on internet communities, according to an exemplary embodiment.

[0073] Figure 3 This is a flowchart illustrating yet another method for recognizing emojis based on an internet community, according to an exemplary embodiment.

[0074] Figure 4 This is a block diagram illustrating an emoji recognition device based on an exemplary embodiment of an internet community.

[0075] Figure 5 This is a block diagram illustrating another internet community-based emoji recognition device according to an exemplary embodiment.

[0076] Figure 6 This is a block diagram illustrating yet another internet community-based emoji recognition device according to an exemplary embodiment. Detailed Implementation

[0077] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0078] Figure 1 This is a flowchart illustrating an emoji recognition method based on an exemplary embodiment of an internet community.

[0079] like Figure 1 As shown, according to a first aspect of the present invention, a method for recognizing emojis based on internet communities is provided, the method comprising:

[0080] Step S101: Obtain images from internet communities and classify the images using a pre-trained image classification model to determine whether the images belong to the emoji category;

[0081] The emoji packs mainly include: natural emoji packs and handmade emoji packs. Natural emoji packs mainly refer to emoji packs made from photos, while handmade emoji packs mainly refer to emoji packs made by hand based on some anime characters or emoji expressions.

[0082] Step S102: When the image does not belong to the emoji category, obtain the text detection result in the image through the OCR model;

[0083] When the image belongs to the meme category, the image is added to a meme recommendation pool.

[0084] In step S103, it is determined whether the text detection result contains a keyword in a preset keyword library; the preset keyword library can be a sensitive word library.

[0085] In step S104, when the text detection result contains a keyword, it is determined whether the text detection result meets a preset filtering condition.

[0086] In one embodiment, preferably, the preset filtering condition includes:

[0087] An effective matching ratio of the text detection result is greater than or equal to a first preset threshold value;

[0088] An IOU of a text box of the text detection result and a preset user ID watermark box is less than a second preset threshold value.

[0089] In one embodiment, preferably, the effective matching ratio is calculated by using the following first calculation formula:

[0090] Effective matching ratio = maximum matching text length / total text length in the text detection result;

[0091] The IOU of the text box of the text detection result and the preset user ID watermark box is calculated by using the following second calculation formula:

[0092] IOU = intersection area of the text box and the preset user ID watermark box / union area of the text box and the preset user ID watermark box.

[0093] In step S105, when the text detection result meets the preset filtering condition, it is determined that the image belongs to the meme category, and the image is added to the meme recommendation pool.

[0094] In this embodiment, by filtering the text box, the problem of false injury caused by sensitive word matching can be reduced, and the preset filtering condition is used to recall the meme that does not belong to the general type and does not contain text.

[0095] Figure 2 is a flowchart of another Internet community-based meme recognition method according to an exemplary embodiment.

[0096] As shown in Figure 2 In one embodiment, preferably, the method further includes:

[0097] In step S201, an image MD5 feature library is obtained.

[0098] Step S202, determine whether the target image matching the image can be found in the image MD5 feature library;

[0099] Step S203, when the target image is not found, monitor whether the image is sent again within a first preset time period;

[0100] Step S204, when the image is sent again within the first preset time period, record the time t1 of the image being sent again;

[0101] Step S205, monitor the first number of times the image is sent within a second preset time period after the time t1 of being sent again;

[0102] Step S206, when the first number of times is greater than or equal to a first preset number of times, recommend in the post comment emoticon package recommendation, and record the second number of times the image is sent within (t1+second preset time period) to (t1+third preset time period);

[0103] Step S207, when the second number of times is greater than or equal to the first preset number of times, add the image to the emoticon package recommendation pool.

[0104] In this embodiment, a user behavior data based method is used to discover emoticon package images. In a specific embodiment, first, an md5 feature library of all images is established. For a new image, if the md5 is matched in the feature library, it is not concerned. If the md5 is not matched in the feature library, the image needs to be concerned about the re-sending situation within 48 hours. If it is not re-sent within 48 hours, it is not concerned. If it is re-sent within 48 hours, the time t1 of being re-sent is recorded. The number of times of being sent again from (t1, t1+24 hours) is concerned. If the number of times (s1) reaches a threshold T1, the image is recommended in the post comment emoticon package recommendation from (t1+24 hours, t1+48 hours). The weight is allocated according to s1. The number of times of being sent within (t1+24 hours, t1+48 hours) is recorded. If it reaches a threshold T2, the image is considered as an emoticon package and put into an emoticon package pool for recommendation sorting.

[0105] Figure 3 is a flowchart of another kind of internet community based emoticon package recognition method according to an example embodiment.

[0106] As Figure 3 shown, in one embodiment, preferably, the method further comprises:

[0107] Step S301, obtain image materials in historical emoticon package topics and historical image comment data, and perform emoticon package category labeling to obtain a labeled training set;

[0108] Step S302, training is performed using the training set and the EfficientNet model to obtain the pre-trained image classification model.

[0109] In this embodiment, for training an image classifier for classifying various types of meme materials, image materials in the "meme" topic in the past 60 days are collected, and image comments in the past 90 days are sampled and labeled, thereby completing the construction of an image material classification data set, wherein the meme categories include: cute pet memes, cute baby memes, star memes, animation memes, and simple memes. Then, the image material classifier is trained based on EfficientNet using this data set.

[0110] Figure 4 is a block diagram of an Internet community-based meme recognition device according to an exemplary embodiment.

[0111] As Figure 4 shown, according to a second aspect of an embodiment of the present application, an Internet community-based meme recognition device is provided, which includes:

[0112] The classification module 41 is configured to obtain images in the Internet community and use a pre-trained image classification model to classify the images to determine whether the images belong to the meme category.

[0113] The recognition module 42 is configured to, when the images do not belong to the meme category, obtain a text detection result in the images by an OCR model.

[0114] The first determination module 43 is configured to determine whether the text detection result contains a keyword in a preset keyword library.

[0115] The second determination module 44 is configured to, when the text detection result contains a keyword, determine whether the text detection result satisfies a preset filtering condition.

[0116] The third determination module 45 is configured to, when the text detection result satisfies the preset filtering condition, determine that the images belong to the meme category, and add the images to a meme recommendation pool.

[0117] In one embodiment, preferably, the preset filtering condition includes:

[0118] The effective matching proportion of the text detection result is greater than or equal to a first preset threshold value.

[0119] The IOU of the text box of the text detection result and the preset user ID watermark box is less than a second preset threshold value.

[0120] In one embodiment, preferably, the effective matching ratio is calculated by using the following first calculation formula:

[0121] Effective matching ratio = maximum matching text length / total length of text in text detection result

[0122] The IOU of the text box of the text detection result and the preset user ID watermark box is calculated by using the following second calculation formula:

[0123] IOU = intersection area of text box and preset user ID watermark box / union area of text box and preset user ID watermark box

[0124] Figure 5 is a block diagram of another Internet community-based emoticon package recognition device according to an exemplary embodiment.

[0125] As shown in Figure 5 one embodiment, preferably, the device further comprises:

[0126] A first acquisition module 51 is configured to acquire an image MD5 feature library.

[0127] A fourth determination module 52 is configured to determine whether a target image matching the image can be found in the image MD5 feature library.

[0128] A first monitoring module 53 is configured to monitor whether the image is sent again within a first preset time period when the target image is not found.

[0129] A first recording module 54 is configured to record a time t1 when the image is sent again within the first preset time period.

[0130] A second monitoring module 55 is configured to monitor a first number of times the image is sent within a second preset time period after the time t1 when the image is sent again.

[0131] A second recording module 56 is configured to recommend in a post comment emoticon package recommendation when the first number of times is greater than or equal to a first preset number of times, and record a second number of times the image is sent within a third preset time period after the time t1+ second preset time period.

[0132] A processing module 57 is configured to add the image to an emoticon package recommendation pool when the second number of times is greater than or equal to the first preset number of times.

[0133] Figure 6 is a block diagram of another Internet community-based emoticon package recognition device according to an exemplary embodiment.

[0134] As shown inFigure 6 As shown, in one embodiment, preferably, the device further comprises:

[0135] The second acquisition module 61 is configured to acquire image materials and historical image comment data in a historical meme topic, and perform meme category labeling to obtain a labeled training set.

[0136] The training module 62 is configured to train the EfficientNet model using the training set to obtain the pre-trained image classification model.

[0137] In one embodiment, preferably, the processing module is further configured to:

[0138] When the image belongs to the meme category, the image is added to a meme recommendation pool.

[0139] According to a third aspect of an embodiment of the present application, there is provided a meme recognition device based on an Internet community, the device comprising:

[0140] a processor;

[0141] a memory for storing processor-executable instructions;

[0142] The processor is configured to:

[0143] acquire an image in the Internet community and use a pre-trained image classification model to classify the image to determine whether the image belongs to a meme category;

[0144] When the image does not belong to the meme category, acquire a text detection result in the image through an OCR model;

[0145] determine whether the text detection result contains a keyword in a preset keyword library;

[0146] When the text detection result contains the keyword, determine whether the text detection result satisfies a preset filtering condition;

[0147] When the text detection result satisfies the preset filtering condition, determine that the image belongs to the meme category, and add the image to a meme recommendation pool.

[0148] According to a fourth aspect of an embodiment of the present application, there is provided a computer-readable storage medium having computer instructions stored thereon, the instructions being executed by a processor to implement the steps of the method according to any one of the embodiments of the second aspect.

[0149] It should be further understood that "a plurality" or "a plurality of something" refers to two or more of that thing, and not to one of that thing. Similarly, "a number of" refers to a non-specific amount of that thing. Such a number can be "one" or greater. In other words, a plurality or number of items refers to one or more items, not one.

[0150] It should be further understood that the terms "first", "second", etc. are used to describe various information but do not imply these are the only two itemsmodifying these terms. These terms are merely used as identifiers for simplicity, and are in no way meant to limit the scope of the application. It is also to be understood that the use of "a", "an", "the", etc., does not exclude the presence of more than one of the referred to item, unless specifically stated otherwise.

[0151] It should be further understood that the processes described with regard to the figures can be implemented as a routine executed using one or more programming languages, which can be implemented as machine, firmware, or software, in combination with hardware that can be implemented as a programmable gate array, application specific integrated circuit (ASIC), programmed processor, or other device. The various processes described herein can be implemented on the same device or different devices.

[0152] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0153] It is to be understood that the application is not limited to the specific structures described herein and illustrated in the accompanying drawings, which provide a specific exemplification of a preferred embodiment of the application. The scope of the application is limited only by the claims.

Claims

1. A method for recognizing emojis based on internet communities, characterized in that, The method includes: Images from online communities are acquired, and a pre-trained image classification model is used to classify the images to determine whether they belong to the emoji category. When the image does not belong to the emoji category, the text detection result in the image is obtained through the OCR model; Determine whether the text detection result contains keywords from a preset keyword library; When the text detection result contains keywords, determine whether the text detection result meets the preset filtering conditions; When the text detection result meets the preset filtering conditions, the image is determined to belong to the emoji category, and the image is added to the emoji recommendation pool; The preset filtering conditions include: The effective matching ratio of the text detection result is greater than or equal to the first preset threshold. The IOU between the text box of the text detection result and the preset user ID watermark box is less than the second preset threshold. The effective matching ratio is calculated using the following first calculation formula: Effective match ratio = maximum matched text length / total text length in the text detection results; The IOU between the text box of the text detection result and the preset user ID watermark box is calculated using the following second calculation formula: IOU = Intersection area of ​​the text box and the preset user ID watermark box / Union area of ​​the text box and the preset user ID watermark box.

2. The method according to claim 1, characterized in that, The method further includes: Obtain the image MD5 feature library; Determine whether a target image matching the image can be found in the image MD5 feature library; If the target image is not received, monitor whether the image is resent within a first preset time period; When the image is retransmitted within the first preset time period, the time t1 when it is retransmitted is recorded. Monitor the first number of times the image is transmitted within a second preset time period after the retransmission time t1; When the first number is greater than or equal to the first preset number, it is recommended in the post comment emoticon recommendation, and the second number of times the image is sent within t1 + second preset time period to t1 + third preset time period is recorded; When the second count is greater than or equal to the first preset count, the image is added to the emoji recommendation pool.

3. The method according to claim 1, characterized in that, The method further includes: Obtain image materials and historical image comment data from historical meme topics, and label them according to meme categories to obtain a labeled training set; The pre-trained image classification model is obtained by training the model using the training set and the EfficientNet model.

4. The method according to claim 1, characterized in that, The method further includes: If the image belongs to the emoji category, the image is added to the emoji recommendation pool.

5. An emoji recognition device based on internet communities, characterized in that, The device includes: The classification module is used to acquire images from internet communities and classify the images using a pre-trained image classification model to determine whether the images belong to the emoji category; The recognition module is used to obtain the text detection result in the image through an OCR model when the image does not belong to the emoji category; The first determining module is used to determine whether the text detection result contains keywords from a preset keyword library; The second determining module is used to determine whether the text detection result meets the preset filtering conditions when the text detection result contains keywords. The preset filtering conditions include: The effective matching ratio of the text detection result is greater than or equal to the first preset threshold. The IOU between the text box of the text detection result and the preset user ID watermark box is less than the second preset threshold. The effective matching ratio is calculated using the following first calculation formula: Effective match ratio = maximum matched text length / total text length in the text detection results; The IOU between the text box of the text detection result and the preset user ID watermark box is calculated using the following second calculation formula: IOU = Intersection area of ​​the text box and the preset user ID watermark box / Union area of ​​the text box and the preset user ID watermark box; The third determining module is used to determine that the image belongs to the emoji category when the text detection result meets the preset filtering conditions, and to add the image to the emoji recommendation pool.

6. An emoji recognition device based on internet communities, characterized in that, The device includes: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Images from online communities are acquired, and a pre-trained image classification model is used to classify the images to determine whether they belong to the emoji category. When the image does not belong to the emoji category, the text detection result in the image is obtained through the OCR model; Determine whether the text detection result contains keywords from a preset keyword library; When the text detection result contains keywords, determine whether the text detection result meets the preset filtering conditions; The preset filtering conditions include: The effective matching ratio of the text detection result is greater than or equal to the first preset threshold. The IOU between the text box of the text detection result and the preset user ID watermark box is less than the second preset threshold. The effective matching ratio is calculated using the following first calculation formula: Effective match ratio = maximum matched text length / total text length in the text detection results; The IOU between the text box of the text detection result and the preset user ID watermark box is calculated using the following second calculation formula: IOU = Intersection area of ​​the text box and the preset user ID watermark box / Union area of ​​the text box and the preset user ID watermark box; When the text detection result meets the preset filtering conditions, the image is determined to belong to the emoji category, and the image is added to the emoji recommendation pool.

7. A computer-readable storage medium storing computer instructions thereon, characterized in that, When executed by the processor, this instruction implements the steps of the method described in any one of claims 1-4.

Images