Message identification method and device

Abstract

The invention provides a message identification method and device, and the method comprises the steps: obtaining a to-be-identified message sent by a user, and determining the message type of the to-be-identified message; based on the message category, determining an identification mode corresponding to the to-be-identified message; and identifying the to-be-identified message based on the identification mode to obtain a message identification result. According to the embodiment, different types of to-be-identified messages can be identified in a targeted manner, and the accuracy of message identification is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a message identification method and device. Background Art

[0002] By identifying messages sent by users, we can effectively avoid the spread of bad news, maintain a good community environment and effectively improve the user experience of other users. The existing technology usually directly intercepts messages when it identifies key features in the message; or sets a certain type of message to be intercepted without identifying the specific content of the message. This results in low accuracy and pertinence of message identification. Summary of the invention

[0003] In view of this, the embodiments of the present disclosure at least provide a message identification method, device, electronic device and storage medium, which can improve the accuracy and pertinence of message identification.

[0004] In a first aspect, an embodiment of the present disclosure provides a message identification method, including:

[0005] Obtain the message to be identified sent by the user, and determine the message category of the message to be identified;

[0006] Based on the message category, determining an identification method corresponding to the message to be identified;

[0007] The message to be identified is identified based on the identification method to obtain a message identification result.

[0008] Optionally, the message category of the message to be identified is a text category; and identifying the message to be identified based on the identification method includes:

[0009] Segment the message to be recognized to obtain word fragments corresponding to the message to be recognized;

[0010] Each word fragment is vectorized to obtain a local feature corresponding to each word fragment; the local feature includes the word fragment's own features and the context features of the word fragment;

[0011] Summarize the local features to obtain the global features corresponding to the message to be identified;

[0012] Perform linear transformation on the global features and calculate the probability of the message to be identified under each candidate classification; take the candidate classification with the highest probability as the target classification of the message to be identified, and determine the recognition result of the message to be identified based on the target classification.

[0013] Optionally, the message category of the message to be identified is an image category; and identifying the message to be identified based on the identification method includes:

[0014] According to the binary image obtained based on the message to be identified, the target area in the message to be identified is located, and the endpoint coordinates of the target area are determined;

[0015] Based on the endpoint coordinates, matrix transformation is performed on the target area to obtain image information corresponding to the target area;

[0016] Parsing the image information, and judging whether the message to be identified includes the information code according to the parsing result;

[0017] In response to the information code being included in the message to be identified, the identification result of the message to be identified is determined according to the parsing result of the image information corresponding to the information code.

[0018] Optionally, the method further comprises:

[0019] In response to the message to be identified not including the information code, preprocessing the message to be identified, and marking the text position in the message to be identified in the preprocessed message to be identified;

[0020] Based on the text position in the message to be identified and the pre-obtained context association information, identifying the text information in the message to be identified;

[0021] Standardize the text information to unify the text format of the text information;

[0022] The text information after the standardization process is classified, and the recognition result of the message to be recognized is determined based on the classification result of the text information.

[0023] Optionally, the message category of the message to be identified is a mini-program category; and identifying the message to be identified based on the identification method includes:

[0024] Extracting text information and image information from the message to be identified;

[0025] Performing vectorization processing on the text information to obtain the text features of the message to be identified, and performing vectorization processing on the image information to obtain the image features of the message to be identified;

[0026] Linear transformation is performed on text features and image features to calculate the probability of the message to be identified under each candidate classification; the candidate classification with the highest probability is used as the target classification of the message to be identified, so as to determine the recognition result of the message to be identified based on the target classification.

[0027] Optionally, the message category of the message to be identified also includes other categories, and the method includes:

[0028] Intercept messages whose message categories are other categories of messages to be identified; among which, messages whose message categories are other categories of messages to be identified are messages to be identified except for messages to be identified in text categories, image categories and mini-program categories.

[0029] In a second aspect, an embodiment of the present disclosure provides a message identification device, including:

[0030] An acquisition module is used to acquire the message to be identified sent by the user and determine the message category of the message to be identified;

[0031] A determination module, used to determine an identification method corresponding to the message to be identified based on the message category;

[0032] The identification module is used to identify the message to be identified based on the identification method to obtain a message identification result.

[0033] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the above-mentioned first aspect, or the steps in any optional implementation of the first aspect are performed.

[0034] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect, or any optional implementation of the first aspect are executed.

[0035] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program, which implements the method of any of the above embodiments when executed by a processor.

[0036] Any of the above aspects or any implementation methods of any of the aspects, by determining the identification method of the message to be identified according to the message category of the message to be identified, and using the corresponding identification method to identify the message, can identify different types of messages in a targeted manner. Since different types of messages have different formats and content characteristics, targeted identification methods can better adapt to the characteristics of different types of messages, which helps to improve the accuracy of message identification. In addition, by setting targeted identification methods for different types of messages, the recognition efficiency of identifying different types of messages can also be improved. Furthermore, when the identification method for identifying a certain type of message changes, or when a new identification method for a certain type of message is to be added, only the identification method for that type can be modified or added, without affecting the identification methods for other types of messages, thereby improving the flexibility of message identification.

[0037] The effects of the above-mentioned message identification device, electronic device and storage medium can be found in the description of the above-mentioned message identification method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0039] Figure 1 A flow chart of a message identification method provided by an embodiment of the present disclosure is shown;

[0040] Figure 2 A schematic diagram of a flow chart of a message identification method provided by an embodiment of the present disclosure is shown;

[0041] Figure 3 A schematic diagram of a process of using a model to perform message identification in a message identification method provided by an embodiment of the present disclosure is shown;

[0042] Figure 4 A structural schematic diagram of a text recognition model in the message recognition method provided by an embodiment of the present disclosure is shown;

[0043] Figure 5 A schematic diagram of extracting text information from a message to be identified of an image category in a message identification method provided by an embodiment of the present disclosure is shown;

[0044] Figure 6 A schematic diagram of a multimodal recognition model in a message recognition method provided by an embodiment of the present disclosure is shown;

[0045] Figure 7 A schematic diagram of a message identification device provided by an embodiment of the present disclosure is shown;

[0046] Figure 8 An exemplary system architecture is shown in which embodiments of the present disclosure may be applied;

[0047] Fig. 9 A schematic diagram of the structure of a computer system of a terminal device or a server used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0049] It should be noted that in the technical solution of the present invention, the collection, use, storage, sharing and transfer of user personal information involved are in compliance with the provisions of relevant laws and regulations, and it is necessary to inform the user and obtain the user's consent or authorization. When applicable, the user's personal information is de-identified and / or anonymized and / or encrypted.

[0050] The raising of the above problems and the solutions are the results obtained by the inventor after practice and careful research. The discovery process of the above problems and the solutions proposed for the above problems should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0051] The technical solutions in the present disclosure will be clearly and completely described below in conjunction with the drawings in the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure claimed for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0052] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0053] To facilitate understanding of this embodiment, a message identification method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the message identification method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device. The terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the message identification method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0054] See also Figure 1 FIG. 1 is a flowchart of a message identification method provided by an embodiment of the present disclosure, and the method includes S101 to S103, wherein:

[0055] S101: Acquire a message to be identified sent by a user, and determine a message category of the message to be identified.

[0056] Figure 2 A flow chart of a message identification method provided by an embodiment of the present disclosure. Figure 2 As shown, the messages to be identified sent by the user can be obtained first. Specifically, the execution subject of the embodiment of the present disclosure can monitor the messages to be identified sent by each user; or the messages to be identified sent by the user can be captured by setting a log recording function. The embodiment of the present disclosure does not specifically limit the specific method for obtaining the messages to be identified, which is subject to the ability to achieve its functions. After obtaining the messages to be identified, the message category of the messages to be identified can be determined.

[0057] In a specific implementation, after obtaining the message to be identified sent by the user, the message category of the message to be identified can be determined by analyzing the message header information. The message category here can include text category, image category, applet category, audio category, positioning category, etc. Messages usually contain metadata, such as file type, MIME (Multipurpose Internet Mail Extensions type, Multipurpose Internet Mail Extensions type) type, etc., and the message category of the message to be identified can be determined by analyzing these metadata. Exemplarily, the message to be identified with the MIME type of "text / plain" is a text category, the message to be identified with the MIME type of "image / jpeg" is an image category, and the message to be identified with the "audio / mpeg" is an audio category, etc. Alternatively, the message category of the message to be identified can also be determined using the interface provided by the platform where the message is located. When obtaining the message to be identified sent by the user on a specific message platform, the interface provided by the platform can usually directly return the message category of the message, thereby determining the message category of the message to be identified. It should be noted that the above-mentioned method of determining the message category of the message to be identified is only used as an example of a possible implementation method of the embodiment of the present disclosure, and does not constitute an improper limitation of the present invention. In practical applications, the message category of the message to be identified can also be determined by analyzing the extension of the message file, pattern matching or pre-trained classification algorithm. The embodiment of the present disclosure does not make specific limitations on this, and the ability to achieve its functions shall prevail.

[0058] In addition, since different types of messages to be identified have different formats, different types of messages can be stored in different storage locations, so that when the messages to be identified are subsequently identified, the messages to be identified can be called from the corresponding storage locations. For example, for text-type messages to be identified, they can be stored in a MySql database; image-type messages to be identified can be stored in OSS cloud storage, etc. It should be noted that the embodiment of the present disclosure does not specifically limit the selection of storage locations, and in actual applications, it can be selected according to actual conditions and needs.

[0059] S102: Determine an identification method corresponding to the message to be identified based on the message category.

[0060] like Figure 2 As shown, in the embodiment of the present disclosure, different identification methods can be set for messages of different categories. After the message category of the message to be identified is determined, the identification method corresponding to the message category can be used to identify it.

[0061] In the embodiments of the present disclosure, recognition models corresponding to various message categories can be pre-trained. After the message category of the message to be identified is determined, the model corresponding to the message category can be used to identify the message to be identified. It should be noted that the message recognition model mentioned here can be embedded in the execution subject of the embodiments of the present disclosure after training. When performing message recognition, the execution subject of the embodiments of the present disclosure can directly use the corresponding model to identify it; or the trained model can be associated with the execution subject of the embodiments of the present disclosure. After the message category of the message to be identified is determined, the model corresponding to the message category can be called to identify the message to be identified.

[0062] In another possible implementation, an identification workflow corresponding to different message categories can also be designed. After the message category of the message to be identified is determined, the message can be identified based on the workflow corresponding to the message category of the message to be identified. It should be noted that the above examples of identification methods are only possible implementation methods of the embodiments of the present disclosure and do not constitute improper limitations on the present invention. In actual applications, they can be set according to actual conditions and requirements to achieve their functions. The steps of identifying the message to be identified using the corresponding identification method are described in detail in S103 and will not be repeated here.

[0063] S103: Identify the message to be identified based on the identification method to obtain a message identification result.

[0064] In a possible implementation manner, the messages to be identified may be identified using various models embedded in the execution subject of the embodiment of the present disclosure. Figure 3 A schematic diagram of a process of using a model to perform message identification in a message identification method provided in an embodiment of the present disclosure. Figure 3As shown, when the message category of the message to be identified is a text category, a pre-trained text recognition model can be used to identify it, and then the recognition result of the message to be identified is determined according to the classification result of the model; when the message category of the message to be identified is an image category, it can be determined whether the image message includes an information code. If the information code is included, the recognition result of the message to be identified can be determined according to the parsing result of the information code; if the information code is not included, text information can be identified from the image message, and then the text information can be identified using the text recognition model to obtain the recognition result of the message to be identified; when the message to be identified is a mini-program category, the text information such as the title text and description text in the mini-program and the image information such as the mini-program logo can be used to determine the recognition result of the message to be identified using a pre-trained multimodal recognition model; for other categories of messages to be identified, such as audio categories, video categories, positioning categories, etc., corresponding models can also be trained to identify them, or messages of these categories can also be directly intercepted, and the embodiments of the present disclosure do not make specific limitations on this.

[0065] It should be noted that the above-mentioned specific process of using the model to identify the message to be identified is only a possible implementation method of the embodiment of the present disclosure. In practical applications, the pre-trained model can also be used to identify the message to be identified in any process or manner. The embodiment of the present disclosure does not make specific restrictions on this, and the function can be realized. The following describes in detail the specific method of using the pre-trained model to identify different categories of messages to be identified.

[0066] In the embodiment of the present disclosure, when the message to be identified is a text message, a pre-trained Figure 4 The text recognition model shown is used to recognize it. Specifically, the message to be recognized is recognized based on the recognition method, including: segmenting the message to be recognized to obtain each word fragment corresponding to the message to be recognized; vectorizing each word fragment to obtain local features corresponding to each word fragment; the local features include the own features of the word fragment and the context features of the word fragment; summarizing each local feature to obtain the global feature corresponding to the message to be recognized; performing linear transformation on the global feature to calculate the probability of the message to be recognized under each candidate classification; taking the candidate classification with the highest probability as the target classification of the message to be recognized, so as to determine the recognition result of the message to be recognized based on the target classification.

[0067] In the specific implementation, Figure 4 As shown in FIG. 1 , the message to be identified can be split according to the part of speech, the number of characters, etc., thereby obtaining multiple word fragments of the message to be identified. Figure 4 As shown, a CLS tag can be added before the first word segmentation to aggregate the global features of the entire message to be identified in the subsequent vectorization process. Figure 4 The PAD marker in represents a padding marker. During the training and inference process, in order to ensure that each input sequence has the same length, it is usually necessary to pad the shorter sequences. The PAD marker is used to fill these sequences so that the length of all input sequences remains consistent. Figure 4 The position marked by "0" in the padding layer is the position filled with the PAD tag. After obtaining each word fragment, each word fragment will be passed to the multi-layer Transformer encoder for vectorization processing. In each layer, each word fragment can interact and integrate with the vectors of all other word fragments, thereby capturing the global features and local semantic features in the text, so that each word fragment contains not only its own local features, but also its features in the entire context. After passing through multiple layers of Transformer encoders, the local features corresponding to each word fragment (including its own features and contextual features) will be integrated and summarized into the vector of the CLS tag to identify the global features of the message to be identified. The global features can then be passed to the following: Figure 4 Assume that the linear layer includes nine neurons, corresponding to nine possible text classification results. The linear layer can be used to perform linear transformation on the global features and calculate the unnormalized score corresponding to each classification. The specific calculation formula can be shown as follows:

[0068] ;

[0069] Among them, logits indicates the calculated unnormalized score, W represents the weight matrix of the linear layer, H represents the global feature, and b represents the bias term. After obtaining the unnormalized score corresponding to each category, the Softmax function can be used to convert it into a probability distribution so that the sum of each logits is 1. The specific calculation formula can be as follows:

[0070] ;

[0071] in, represents the unnormalized score of the kth category; K represents the number of categories; Indicates the probability that the message to be identified belongs to the kth classification. Thus, the probability of the message to be identified in each possible classification result is obtained, and finally the classification with the highest probability can be used as the classification result of the message to be identified, thereby determining the identification result of the message to be identified.

[0072] In the embodiments of the present disclosure, in order to refine the classification granularity of the message to be recognized, the classification results corresponding to the text recognition model may include various possible classifications such as competing companies, sensitive information, artificial customer service, marketing advertisements, chatting information, product recommendations, etc.; or the classification results may simply be set to positive and negative; the embodiments of the present disclosure do not make specific limitations on the specific classification results, which can be set according to the actual application situation.

[0073] In the embodiments of the present disclosure, after the output layer of the conventional text recognition model, a linear layer is added to undertake the downstream text classification task. Thereby, the high-dimensional features of the message to be recognized can be effectively mapped to the classification space, which not only simplifies the model structure, but also improves the accuracy and efficiency of the classification of the message to be recognized, and further improves the recognition efficiency of the message to be recognized. In addition, when training the above-mentioned text recognition model, the parameters of the model can be fine-tuned according to the current application scenario to make it more suitable for the classification and recognition task in the current scenario, and thus a higher classification accuracy can be achieved.

[0074] In addition to using the model to recognize the message to be recognized of the text category, predefined rules or patterns can also be used to classify the message to be recognized, so as to realize the recognition of the message to be recognized; or, methods such as logistic regression can be used to classify the message to be recognized of the text category to obtain the recognition result of the message to be recognized. The embodiments of the present disclosure do not make specific limitations on this, as long as its function can be realized.

[0075] In the prior art, for the message to be recognized of the text category, the keyword detection method is usually used for recognition. Messages containing preset keywords or sensitive words are usually directly intercepted, which may lead to false detection and missed detection. Exemplarily, assume that Company B has preset the competing company name "Company A" as a sensitive word in advance, and when the sensitive word is detected, the message is intercepted. However, if the message sent by the user is "The service of Company A is not as good as that of Company B", at this time, the system will also intercept the message when it recognizes the sensitive word "Company A", resulting in false detection. Or some users insert other characters into "Company A" in order to avoid the keyword detection technology, such as "A&&*公￥#@司&*# has better service", and at this time, it is also impossible to detect the preset sensitive word "Company A" in the message to be recognized, resulting in missed detection. The embodiments of the present disclosure adopt a natural language understanding algorithm to understand the user's intention for the message to be recognized of the text category, and make fine-tuning on the basis of the conventional natural language understanding model (that is, adding a linear layer for text classification), thereby solving the problems of false detection and missed detection brought by the keyword recognition algorithm, improving the recognition accuracy and efficiency of the message to be recognized, and the text classification obtained based on the linear layer also helps the downstream to take further processing measures for the message to be recognized.

[0076] In the disclosed embodiment, when the message to be identified is an image category, a pre-trained information code recognition model can be used to first determine whether the image to be identified includes an information code, and then determine the recognition result of the message to be identified. Specifically, the message category of the message to be identified is an image category; the message to be identified is identified based on the recognition method, including: locating the target area in the message to be identified based on the binary image obtained based on the message to be identified, and determining the endpoint coordinates of the target area; based on the endpoint coordinates, performing a matrix transformation on the target area to obtain image information corresponding to the target area; parsing the image information, and judging whether the message to be identified includes an information code based on the parsing result; in response to the information code being included in the message to be identified, determining the recognition result of the message to be identified based on the parsing result of the image information corresponding to the information code.

[0077] In the specific implementation, since it is known from the analysis in the actual scene that most of the risk images contain advertising information codes and text information, but there are no obvious abstract features of the image, the pre-trained information code recognition model can be used to determine whether the message to be identified in the image category includes an information code. First, the image of the message to be identified can be converted into a single-channel grayscale image, which can be achieved by setting different weights for the three RGB color channels. For example, the grayscale value of the image to be identified can be calculated using the following formula:

[0078] ;

[0079] in, Represents the gray value of the image to be recognized; Represents the red channel value of the original image to be identified; Represents the green channel value of the original image to be identified; Represents the blue channel value of the original image to be identified. The weights before each color channel can be set according to the actual situation. The above numerical values ​​are only used as an example of a possible implementation method and do not constitute an improper limitation of the present invention. After the grayscale image of the image to be identified is obtained, it can also be binarized to convert the grayscale image into a binary image. Specifically, a grayscale threshold t can be preset, and pixels greater than the threshold can be set to 1, and pixels less than the threshold can be set to 0. The specific formula can be as follows:

[0080] ;

[0081] in, Represents the binarization result of the pixel with coordinates (x, y) in the image to be identified. represents the gray value of the pixel with coordinates (x, y); t represents the preset gray threshold.

[0082] After obtaining the binary image of the image to be identified, the geometric detection algorithm can be used to detect the target area in the image. Here, the shape of the target area is the area where the information code may exist, and its shape can be any shape where the information code may appear, such as square, rectangle or circle. Exemplarily, assuming that the information code is a square or rectangle, the geometric detection algorithm can be used to detect the straight lines in the image, and then the relative position and angle of the combination of four straight lines can be verified to determine whether they form a square or rectangle; if so, the area can be used as the target area where the information code may appear. Alternatively, the target area can be located by detecting whether the positioning pattern of the information code is included in the image. After locating the target area, its endpoint coordinates can be determined; since the image may be distorted, after obtaining the endpoint coordinates of the target area, it can be matrix transformed to obtain a standard area that may include the information code, as well as standard image information in the area. After obtaining the standard image information, you can use conventional information code analysis technology to try to decode it. If the decoding is successful, it means that the image to be identified includes the information code. At this time, you can determine the recognition result corresponding to the message to be identified based on the analysis result of the information code; if the analysis fails, it means that the image to be identified does not include the information code, and then you can further identify the text information in the image to be identified to obtain the recognition result of the message to be identified. The method of extracting the text information in the image to be identified and identifying it is described below and will not be repeated here. It should be noted that the recognition result for the image type message to be identified may also include multiple recognition results such as violation classification and social classification; or it can be simply divided into positive and negative types, and the disclosed embodiment does not make specific limitations on this.

[0083] Alternatively, if the analysis fails, the image information can be further corrected, such as adjusting the grayscale threshold in the binarization process, re-performing edge detection or other image processing steps, so as to improve the success rate of the information code analysis and avoid the problem of inaccurate recognition results of the image to be identified due to the image processing process. In addition, when determining the target area, if a locator including the information code is detected in the image, the subsequent steps can be omitted, and the information code in the target area can be directly extracted and analyzed to determine the recognition result of the message to be identified.

[0084] If it is finally determined that the message to be identified does not include the information code, the text information in the image can be extracted to determine the recognition result of the message to be identified. Specifically, in response to the fact that the message to be identified does not include the information code, the message to be identified is preprocessed, and the text position in the message to be identified is marked in the preprocessed message to be identified; based on the text position in the message to be identified and the pre-obtained context association information, the text information in the message to be identified is identified; the text information is standardized to unify the text format of the text information; the text information after the standardized processing is classified, and the recognition result of the message to be identified is determined based on the classification result of the text information.

[0085] Figure 5 A schematic diagram of extracting text information from a message to be recognized of an image category in a message recognition method provided in an embodiment of the present disclosure. In a specific implementation, the image to be recognized can be input into a pre-trained character recognition model, such as Figure 5 As shown, after receiving the image, the character recognition model can pre-process it according to the actual situation, such as noise reduction, cropping, contrast enhancement, etc., so as to improve the accuracy of subsequent recognition of characters in the image. Then, the position detection algorithm can be used to mark the text area where the text exists in the image, and the text information in the image can be recognized in the text area. When recognizing text information, the long-short time memory structure can be introduced into the character recognition process, the relationship between the previous and next texts can be considered, and the continuity and contextual semantics of the text can be understood. For example, when recognizing a sentence, the character recognition model will consider the recognition results of the previous text to infer the current text, and remember the recognized text, which will help to better understand and predict the next text information. Through contextual association information, the association and dependency between characters can be better handled, and the errors that may occur when recognizing characters independently due to ambiguity of some characters can be reduced. For example, in the logistics scenario, when recognizing the "express order number", if the word "express" has been recognized, then recognizing the "order number" will be more accurate.

[0086] After obtaining the text information in the image, you can also define a regular expression pattern based on the structure and content of the text information, such as extracting email addresses, phone numbers, date formats, etc. Regular expressions can match and extract text information that conforms to a specific pattern, thereby removing unnecessary characters or formats in the text information, normalizing the format of the text information, and helping downstream text analysis tasks to be more efficient and accurate. Once the text information is normalized and extracted, it can be used as input data for the text recognition model mentioned above to determine the recognition result for the message to be recognized.

[0087] In addition, the final text information may be classified by any other method to determine the recognition result of the message to be recognized, such as predefined rules or patterns, logistic regression algorithms, etc.; the embodiments of the present disclosure do not make specific limitations on this, and the functionality can be realized.

[0088] In the disclosed embodiment, when the message category of the message to be identified is a mini-program category, it can be identified using a pre-trained multimodal recognition model. Specifically, the message category of the message to be identified is a mini-program category; the message to be identified is identified based on the identification method, including: extracting text information and image information in the message to be identified; vectorizing the text information to obtain the text features of the message to be identified, and vectorizing the image information to obtain the image features of the message to be identified; performing linear transformation on the text features and the image features, and calculating the probability of the message to be identified under each candidate classification; taking the candidate classification with the highest probability as the target classification of the message to be identified, so as to determine the recognition result of the message to be identified based on the target classification.

[0089] In the specific implementation, since the messages to be identified in the mini-program category include both independent text information and image information, the text information mostly does not appear in the image, and since the text information and image information may be stored in different storage locations, a multimodal recognition model can be trained for this category of messages to be identified, instead of using existing text recognition models, information code recognition models, and character recognition models. Figure 6 A schematic diagram of a multimodal recognition model in the message recognition method provided in the embodiment of the present disclosure. Figure 6 As shown, text information and image information can be extracted from the messages to be recognized in the mini-program category. The specific extraction method is not specifically limited in the embodiment of the present disclosure. The multimodal recognition model can perform vectorization processing on the input text information and image information respectively (i.e., by Figure 6 The Transformer layer and the Residual Block layer in the image are used to obtain the text features and image features of the message to be recognized (i.e. Figure 6 The obtained text features and image features can then be concatenated as global features of the message to be recognized, and then the global features are passed through a linear layer to obtain the classification results of the message to be recognized, and then the final recognition results are determined. The step of obtaining the classification results by passing the global features obtained by concatenating the text features and image features through a linear layer is similar to the step in which the text recognition model obtains the classification results of the message to be recognized of the text category according to the global features in the previous article, and will not be repeated here.

[0090] It should be noted that any of the above-mentioned models can be adjusted according to their specific application scenarios during training to improve the recognition or classification efficiency of the model. The embodiments of the present disclosure do not specifically limit the specific training steps and application scenarios of the model.

[0091] In addition, for messages to be identified other than text categories, image categories, and mini-program categories, different identification methods can be set according to different application scenarios. For example, for scenarios where there are very few messages to be identified in other categories, messages to be identified in other categories can be directly intercepted. In the case where there are a large number of messages to be identified in other categories, corresponding models can also be trained for messages to be identified in other categories to determine the identification results of messages to be identified in this category. The embodiments of the present disclosure do not specifically limit this.

[0092] In the disclosed embodiment, a pre-trained risk assessment model may also be used to evaluate the recognition results of the recognition model, thereby optimizing the model, which helps to further improve the accuracy of recognizing messages to be recognized using the pre-trained model.

[0093] In another possible implementation manner, the message to be identified may also be identified by using a preset workflow associated with the message category of the message to be identified.

[0094] In a specific implementation, different workflows can be set according to different types of messages to be identified. For example, for the same preprocessing, for the messages to be identified in the text category, the preprocessing can be to remove special characters in the text, unify the text font size, delete spaces, etc.; and for the messages to be identified in the image category, the preprocessing can be to denoise the image, improve the contrast, crop, etc. For another example, in the step of feature extraction, for the messages to be identified in the text category, the TF-IDF (Term Frequency-Inverse Document Frequency) algorithm, word embedding algorithm, etc. can be used to extract the features of the text information; and for the messages to be identified in the audio category, the features of the messages to be identified can be extracted using methods such as spectrograms. The other steps are similar and will not be repeated here. In this way, the recognition work column for different categories of messages to be identified can be obtained. After the message category of the message to be identified is determined, it can be identified according to the corresponding workflow.

[0095] It should be noted that the above method of identifying messages using a pre-trained model and a pre-set workflow is only used as an example of a possible implementation method of the present disclosure. In actual applications, it can be set by itself according to actual requests and needs, and the present disclosure embodiment does not make specific limitations on this.

[0096] like Figure 2As shown, in the embodiment of the present disclosure, after determining the identification result of the message to be identified, a corresponding processing result can be taken. For example, the message to be identified that is indicated as a risk message by the identification result can be intercepted, or the user who sends the risk message can be warned. The embodiment of the present disclosure does not specifically limit the subsequent processing steps, and can be set according to actual conditions in actual applications.

[0097] According to a second aspect of the embodiment of the present disclosure, Figure 7 As shown, a message identification device 700 is provided, comprising:

[0098] The acquisition module 701 is used to acquire the message to be identified sent by the user and determine the message category of the message to be identified;

[0099] A determination module 702, configured to determine an identification method corresponding to a message to be identified based on a message category;

[0100] The identification module 703 is used to identify the message to be identified based on the identification method to obtain a message identification result.

[0101] Optionally, the identification module 703 is specifically used for:

[0102] Segment the message to be recognized to obtain word fragments corresponding to the message to be recognized;

[0103] Each word fragment is vectorized to obtain a local feature corresponding to each word fragment; the local feature includes the word fragment's own features and the context features of the word fragment;

[0104] Summarize the local features to obtain the global features corresponding to the message to be identified;

[0105] Perform linear transformation on the global features and calculate the probability of the message to be identified under each candidate classification; take the candidate classification with the highest probability as the target classification of the message to be identified, and determine the recognition result of the message to be identified based on the target classification.

[0106] Optionally, the identification module 703 is specifically used for:

[0107] According to the binary image obtained based on the message to be identified, the target area in the message to be identified is located, and the endpoint coordinates of the target area are determined;

[0108] Based on the endpoint coordinates, matrix transformation is performed on the target area to obtain image information corresponding to the target area;

[0109] Parsing the image information, and judging whether the message to be identified includes the information code according to the parsing result;

[0110] In response to the information code being included in the message to be identified, the identification result of the message to be identified is determined according to the parsing result of the image information corresponding to the information code.

[0111] Optionally, the identification module 703 is specifically used for:

[0112] In response to the message to be identified not including the information code, preprocessing the message to be identified, and marking the text position in the message to be identified in the preprocessed message to be identified;

[0113] Based on the text position in the message to be identified and the pre-obtained context association information, identifying the text information in the message to be identified;

[0114] Standardize the text information to unify the text format of the text information;

[0115] The text information after the standardization process is classified, and the recognition result of the message to be recognized is determined based on the classification result of the text information.

[0116] Optionally, the identification module 703 is specifically used for:

[0117] Extracting text information and image information from the message to be identified;

[0118] Performing vectorization processing on the text information to obtain the text features of the message to be identified, and performing vectorization processing on the image information to obtain the image features of the message to be identified;

[0119] Linear transformation is performed on text features and image features to calculate the probability of the message to be identified under each candidate classification; the candidate classification with the highest probability is used as the target classification of the message to be identified, so as to determine the recognition result of the message to be identified based on the target classification.

[0120] Optionally, the message category of the message to be identified also includes other categories, and the identification module 703 is further used to:

[0121] Intercept messages whose message categories are other categories of messages to be identified; among which, messages whose message categories are other categories of messages to be identified are messages to be identified except for messages to be identified in text categories, image categories and mini-program categories.

[0122] According to the third aspect of an embodiment of the present disclosure, there is provided an electronic device for message identification, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method provided by the first aspect of the embodiment of the present invention.

[0123] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present disclosure is implemented.

[0124] According to a fifth aspect of an embodiment of the present invention, there is provided a computer program product, including a computer program, which implements the method of any of the above embodiments when executed by a processor.

[0125] Figure 8 An exemplary system architecture 800 to which the message identification method or message identification apparatus implemented by the present disclosure can be applied is shown.

[0126] like Figure 8 As shown, system architecture 800 may include terminal devices 801, 802, 803, a network 804, and a server 805. Network 804 is used to provide a medium for communication links between terminal devices 801, 802, 803 and server 805. Network 804 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0127] Users can use terminal devices 801, 802, and 803 to interact with server 805 through network 804 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 801, 802, and 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0128] The terminal devices 801 , 802 , and 803 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0129] The server 805 may be a server that provides various services, such as a background management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 801, 802, and 803. The background management server may process the received message identification request and feed back the processing result (only an example) to the terminal device.

[0130] It should be noted that the message identification method provided in the embodiment of the present invention is generally executed by the server 805, and accordingly, the message identification device is generally set in the server 805. The message identification method provided in the embodiment of the present invention can also be executed by the terminal devices 801, 802, and 803, and accordingly, the message identification device can be set in the terminal devices 801, 802, and 803.

[0131] It should be understood that Figure 8The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0132] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of a terminal device suitable for implementing an embodiment of the present invention. Fig. 9 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0133] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0134] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0135] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0136] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0137] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor. For example, a processor includes an acquisition module, a determination module, and an identification module, wherein the names of these modules do not constitute a limitation on the modules themselves in certain circumstances. For example, the acquisition module may also be described as a "module for acquiring messages to be identified sent by users and determining the message category of the messages to be identified".

[0139] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device implements the following method: obtaining a message to be identified sent by a user, determining the message category of the message to be identified; determining an identification method corresponding to the message to be identified based on the message category; identifying the message to be identified based on the identification method to obtain a message identification result.

[0140] Finally, it should be noted that the above embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A message identification method, characterized in that: include: Obtaining a message to be identified sent by a user, and determining a message category of the message to be identified; Based on the message category, determining an identification method corresponding to the message to be identified; The message to be identified is identified based on the identification method to obtain a message identification result.

2. The method according to claim 1, characterized in that The message category of the message to be identified is a text category; identifying the message to be identified based on the identification method includes: Segmenting the message to be identified to obtain word fragments corresponding to the message to be identified; Performing vectorization processing on each word fragment to obtain local features corresponding to each word fragment; the local features include the self features of the word fragment and the context features of the word fragment; Summarizing the local features to obtain a global feature corresponding to the message to be identified; The global features are linearly transformed to calculate the probability of the message to be identified under each candidate classification; the candidate classification with the highest probability is used as the target classification of the message to be identified, so as to determine the identification result of the message to be identified based on the target classification.

3. The method according to claim 1, characterized in that The message category of the message to be identified is an image category; Identifying the message to be identified based on the identification method includes: Locating a target area in the message to be identified according to a binary image obtained based on the message to be identified, and determining the endpoint coordinates of the target area; Based on the endpoint coordinates, performing matrix transformation on the target area to obtain image information corresponding to the target area; Parsing the image information, and judging whether the message to be identified includes an information code according to the parsing result; In response to the information code being included in the message to be identified, the identification result of the message to be identified is determined according to the parsing result of the image information corresponding to the information code.

4. The method according to claim 3, characterized in that The method further comprises: In response to the message to be identified not including the information code, preprocessing the message to be identified, and marking the text position in the message to be identified in the preprocessed message to be identified; Based on the text position in the message to be identified and the pre-obtained context association information, identifying the text information in the message to be identified; Standardizing the text information to unify the text format of the text information; The text information after the standardization process is classified, and the recognition result of the message to be recognized is determined based on the classification result of the text information.

5. The method according to claim 1, characterized in that The message category of the message to be identified is the mini-program category; Identifying the message to be identified based on the identification method includes: Extracting text information and image information from the message to be identified; Performing vectorization processing on the text information to obtain text features of the message to be identified, and performing vectorization processing on the image information to obtain image features of the message to be identified; The text features and the image features are linearly transformed to calculate the probability of the message to be identified under each candidate classification; the candidate classification with the highest probability is used as the target classification of the message to be identified, so as to determine the recognition result of the message to be identified based on the target classification.

6. The method according to claim 1, characterized in that The message category of the message to be identified also includes other categories, and the method includes: Intercept the messages whose message categories are other categories to be identified; wherein the messages whose message categories are other categories to be identified are messages to be identified except for the messages to be identified of the text category, image category and mini-program category.

7. A message identification device, characterized in that: include: An acquisition module, used to acquire a message to be identified sent by a user, and determine a message category of the message to be identified; A determination module, configured to determine, based on the message category, an identification method corresponding to the message to be identified; The identification module is used to identify the message to be identified based on the identification method to obtain a message identification result.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images