Classification Model Update Method and Corresponding Device, Equipment, Medium

Abstract

The present application discloses a classification model updating method and its corresponding device, equipment, and medium. The method includes: obtaining a reference text that meets a preset timeliness, performing word segmentation on it to extract candidate words; fusing the self-information entropy and mutual information entropy of each candidate word to obtain the information score of each candidate word, and taking the candidate words with information scores higher than a preset threshold as the hit words in the reference text; calculating the matching degree of each hit word with each classification in a preset classification thesaurus, and clustering each hit word into each classification according to the corresponding matching degree; determining the classification with the highest matching degree of each hit word as the classification to which the hit word belongs, and updating the mapping relationship between the hit word, its classification, and the reference text to a corpus; starting the training of the classification model. The present application enables the classification model to continuously introduce its semantic understanding ability of dynamically newly added hot information, thereby improving its classification ability for the security attributes of title texts.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of e-commerce information technology, and in particular, to a method for updating a classification model and its corresponding device, equipment, and medium. Background Art

[0002] As an effective means of information management and application, text classification mainly refers to the process of classifying a large number of texts into one or more categories in sequence according to a certain classification system or standard based on the theme or content of the target document. It plays an important role in supporting aspects such as topic extraction, sentiment analysis, public opinion analysis, spam filtering, intelligent question answering, and recommendation systems.

[0003] In recent years, with the rise of Attention (attention mechanism), pre-trained and fine-tuned models are increasingly used in NLP language tasks. First, the pre-trained model is the result obtained after training based on a large amount of complete semantics, and it has good effects in some complex semantic recognition, question answering, and understanding. Unfortunately, in the cross-border e-commerce field concerned by the present application, most of the content, especially e-commerce titles used to describe e-commerce products, has no complete semantics in the context, and is basically a splicing of keywords. Therefore, directly using the pre-trained models obtained by major research teams cannot meet the text classification requirements in the e-commerce field.

[0004] At the same time, the training process of the deep learning model based on fine-tuning takes a lot of time, and it is extremely prone to overfitting due to the high complexity of the model. During the process, it is necessary to continuously update and iterate the samples, which is time-consuming and laborious. Therefore, when the model is put into actual production for real-time risk control content detection, the model will not be updated in time, resulting in great risks in risk control. Especially in the cross-border e-commerce field, since commodity sales are targeted at different countries and different cultures, it is very easy to involve some prohibited or sensitive points in the other party's culture, laws, or current events. Then, it is necessary to timely detect some real-time updated commodities and content in the cross-border e-commerce field, analyze their relevant information, and judge whether they belong to the safe or unsafe category.

[0005] In order to realize the judgment of the safety attributes of the on-shelf commodities, the existing methods are to match the relevant content with different pre-established type libraries. However, due to the lack of effective technical means, the update of these libraries is usually carried out manually or collected laggingly. Therefore, the update of the libraries in different countries and regions often has a great delay. If we rely on translating the libraries of other countries to achieve it, the obtained native language library will only be a literal translation result with inaccurate semantics itself, which is more likely to cause great misjudgments for the model that relies on keyword matching.

[0006] In summary, for machine learning models used to implement text classification, whether neural network models or traditional machine learning models, when classifying the title texts commonly used in the e-commerce field, especially the cross-border e-commerce field, due to the lag in the update of their word libraries, it is easy to cause phenomena such as inaccurate and untimely classification of the corresponding models, which urgently need to be improved. Summary of the Invention

[0007] An object of the present application is to provide a classification model update method, its corresponding device, computer device and storage medium for at least some of the deficiencies existing in the prior art.

[0008] To solve the above technical problems, a technical solution adopted by the present application is:

[0009] The present application provides a classification model update method, which includes the following steps:

[0010] Obtain a reference text that meets the preset timeliness, and perform word segmentation on it to extract candidate words;

[0011] Fuse the self-information entropy and mutual information entropy of each candidate word to obtain the information score of each candidate word, and use the candidate words with information scores higher than the preset threshold as the hit words in the reference text;

[0012] Calculate the matching degree of each hit word with each classification in the preset classification word library, and cluster each hit word into the respective classifications according to the corresponding matching degree to form the keywords of the classification word library;

[0013] Determine the classification with the highest matching degree of each hit word as the classification to which the hit word belongs, and update the mapping relationship between the hit word, its classification, and the reference text to the corpus;

[0014] Start the training of the classification model for implementing title text classification, so that the classification model uses the reference text in the corpus as the training sample, and uses the mapping relationship between the hit word corresponding to the reference text in the corpus and its classification as the supervision label of the reference text.

[0015] In a preferred embodiment, obtaining a reference text that meets the preset timeliness and performing word segmentation on it to extract candidate words includes the following steps:

[0016] Call the crawler interface to crawl the website information pages belonging to the target geographical area and within the preset time range;

[0017] Extract the text information from the website information pages, and preprocess it in a formatted manner to obtain the reference text;

[0018] Call the word segmenter to perform word segmentation on the reference text to obtain the candidate words.

[0019] In a specific embodiment, the self-information entropy and mutual information entropy of each candidate word are fused to obtain the information score of each candidate word, and the candidate words with information scores higher than a preset threshold are used as the hit words in the reference text, including the following steps:

[0020] Statistical self-information entropy of each candidate word, the self-information entropy is the sum of the left neighbor information entropy and the right neighbor information entropy of the candidate word;

[0021] Statistical mutual information entropy of each candidate word;

[0022] Adapt each candidate word, linearly fuse its self-information entropy and its mutual information entropy with different weights to obtain the information score of the corresponding candidate word;

[0023] Output the candidate words with information scores higher than the preset threshold as the hit words in the reference text.

[0024] In a specific embodiment, calculate the matching degree of each hit word with each classification in the preset classification thesaurus, and cluster each hit word into the respective classifications according to the corresponding matching degree to form the keywords of the classification thesaurus, including the following steps:

[0025] Vectorize the hit words and the keywords in the preset classification thesaurus;

[0026] According to the word vectors obtained by vectorization, calculate the average cosine similarity between each hit word and the keywords of each classification in the classification thesaurus as the matching degree;

[0027] Store the corresponding relationship data of the matching degree of each hit word with the respective classifications into the classification thesaurus, so that the hit word constitutes the keyword therein, and complete the clustering of the hit words.

[0028] In a further extended embodiment, storing the corresponding relationship data of the matching degree of each hit word with the respective classifications into the classification thesaurus, so that the hit word constitutes the keyword therein, and completing the clustering of the hit words, further includes the following steps:

[0029] Call the classification model for realizing the classification of the title text to classify the reference text, and determine the specific classification to which the reference text belongs in the classification structure of the preset classification thesaurus;

[0030] Compare whether the specific classification of the reference text is consistent with the classification with the highest matching degree of the hit words of the reference text. When the two are inconsistent, recall the corresponding hit words from the preset classification thesaurus;

[0031] Receive the re-annotation data for the matching degree of the recalled hit words, and store it into the preset classification thesaurus.

[0032] In a further extended embodiment, after the classification model is trained to the convergence state and starts running, the following steps are included:

[0033] Obtain the title text input from the outside.

[0034] Segment and vectorize the title text to obtain title word vectors.

[0035] Classify the title text according to the title word vectors to obtain the scoring values of each classification in the classification thesaurus to which it belongs.

[0036] Output the classification label of the classification with the largest scoring value.

[0037] In a preferred embodiment, the title text is in the form of a title of an e-commerce product, including multiple semantically related but grammatically discrete words for describing the e-commerce product.

[0038] To solve the above technical problems, another technical solution adopted by this application is:

[0039] This application provides a classification model updating device, which includes a text reference module, a text scoring module, a classification updating module, a corpus updating module, and a model starting module. Among them, the text reference module is used to obtain reference texts that meet the preset timeliness, and segment them to extract candidate words; the text scoring module is used to fuse the self-information entropy and mutual information entropy of each candidate word to obtain the information score of each candidate word, and use the candidate words with information scores higher than the preset threshold as the hit words in the reference text; the classification updating module is used to calculate the matching degree of each hit word with each classification in the preset classification thesaurus, and cluster each hit word into the respective classifications according to the corresponding matching degree to form the keywords of the classification thesaurus; the corpus updating module is used to determine the classification with the highest matching degree of each hit word as the classification to which the hit word belongs, and update the mapping relationship between the hit word, its classification, and the reference text to the corpus; the model starting module is used to start the training of the classification model for realizing title text classification, use the reference text in the corpus as the training sample, and use the mapping relationship between the hit word corresponding to the reference text in the corpus and its classification as the supervision label of the reference text.

[0040] In a preferred embodiment, the text reference module includes: a crawler scraping sub-module, which is used to call the crawler interface to scrape the website information pages belonging to the target geographical area and within the preset time range; a text extraction sub-module, which is used to extract the text information from the website information pages, and format and preprocess it into the reference text; a text segmentation sub-module, which is used to call the word segmenter to segment the reference text to obtain the candidate words.

[0041] In a specific embodiment, the text scoring module includes: an information entropy sub-module for calculating the self-information entropy of each candidate word, where the self-information entropy is the sum of the left adjacent information entropy and the right adjacent information entropy of the candidate word; a mutual information entropy sub-module for calculating the mutual information entropy of each candidate word; a score fusion sub-module for adapting each candidate word, linearly fusing its self-information entropy and its mutual information entropy with different weights to obtain the information score of the corresponding candidate word; and a candidate word bubbling sub-module for outputting the candidate words with information scores higher than a preset threshold as the hit words in the reference text.

[0042] In a specific embodiment, the classification update module includes: a vector mapping sub-module for vectorizing the hit words and the keywords in the preset classification word library; a matching calculation sub-module for calculating the average cosine similarity between each hit word and the keywords of each classification in the classification word library as the matching degree according to the word vectors obtained by vectorization; and a clustering update sub-module for storing the corresponding relationship data between each hit word and the matching degrees of each classification in the classification word library, so that the hit word constitutes the keyword therein, and completing the clustering of the hit word.

[0043] In a further extended embodiment, the classification update module further includes: a recall classification sub-module for classifying the reference text by calling the classification model for realizing the classification of the title text, and determining the specific classification to which the reference text belongs in the classification structure of the preset classification word library; a comparison and recall sub-module for comparing whether the specific classification of the reference text is consistent with the classification with the highest matching degree of the hit words of the reference text, and when the two are inconsistent, recalling the corresponding hit words from the preset classification word library; and a label refreshing sub-module for receiving the re-annotation data of the matching degree of the recalled hit words and storing them in the preset classification word library.

[0044] In a further extended embodiment, the classification model includes: a title acquisition sub-module for acquiring the externally input title text; a title vector sub-module for segmenting and vectorizing the title text to obtain title word vectors; a title scoring sub-module for classifying the title text according to the title word vectors to obtain the scoring values of its belonging to each classification in the classification word library; and a title classification sub-module for outputting the classification label of the classification with the largest scoring value.

[0045] In a preferred embodiment, the title text is in the form of a title of an e-commerce product, including a plurality of semantically related but grammatically discrete words for describing the e-commerce product.

[0046] To solve the above technical problems, the present application also provides a computer device, including a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor is caused to execute the steps of the classification model updating method.

[0047] To solve the above technical problems, an embodiment of the present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the steps of the classification model updating method.

[0048] Compared with the prior art, the present application has the following advantages:

[0049] The present application takes into account the particularity of the content of e-commerce titles used to describe e-commerce products in terms of grammar and semantics, adapts to the needs of a classification model for classifying and discriminating title texts of such titles, and improves the updating process of such a classification model. Specifically, it realizes the automatic acquisition and automatic updating of the content of the corpus and classification vocabulary required for such a classification model, so that the corpus and classification vocabulary of such a classification model can automatically obtain reference texts from external public resources for a long time, discover new words from them, give classification information related to the new words, fill the classification information of the new words and their reference texts into the corpus required for the classification model, and drive the updated corpus of the classification model to perform model update training, so that the classification model continuously expands its new word recognition ability and can adapt to information security specifications related to different countries, different cultures, and different current affairs hotspots, thereby ensuring that the classification model can timely identify target words such as sensitive words and illegal words in e-commerce titles and continuously dynamically improve the information security recognition ability of the e-commerce platform.

[0050] Since the data sources for new word discovery in the present application can be obtained in various forms, especially by obtaining reference texts that meet the timeliness requirements from the public network, these reference texts can timely reflect sensitive information related to current affairs hotspots, current policies, current laws and regulations, etc. Therefore, it is ensured that the newly discovered words (hit words) are valuable for monitoring security information related to social hotspots. Further, since the self-information entropy and mutual information entropy of candidate words are integrated to examine the information value of candidate words during new word mining, the mined hit words have a high accuracy rate. Therefore, it is comprehensively ensured that hit words related to social hotspots can be timely and accurately extracted to enhance the information security risk control ability of the classification model of the present application.

[0051] Furthermore, applying the technical solution of the present application to the identification of illegal products in the cross-border e-commerce field can continuously and rapidly improve the pre-training corpus of relevant classification models through new word discovery according to the characteristics of different countries and different cultures, quickly update its samples, timely iterate the classification model, rapidly shorten the update time of the classification model, greatly improve the detection accuracy of potential risk information, and minimize the human time cost required for training sample labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The above and / or additional aspects and advantages of the present application will become apparent and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0053] Figure 1 is a schematic flowchart of the basic process of the classification model update method of the present application;

[0054] Figure 2 is a schematic flowchart of the process of obtaining reference texts in the classification model update method of the present application;

[0055] Figure 3 is a schematic flowchart of the process of calculating information scores in the classification model update method of the present application;

[0056] Figure 4 is a schematic flowchart of the process of clustering hit words in the classification model update method of the present application;

[0057] Figure 5 is a schematic flowchart of the process of recalling and labeling hit words in the classification model update method of the present application;

[0058] Figure 6 is a schematic flowchart of the working process of the classification model of the present application;

[0059] Figure 7 is a schematic diagram of the basic structure of the classification model update device of the present application;

[0060] Figure 8 is a basic structural block diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the drawings, where the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and cannot be construed as limiting the present application.

[0062] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application means the presence of the stated features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more of the associated listed items.

[0063] Those skilled in the art can understand that, unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as the general understanding of those of ordinary skill in the art to which the present application pertains. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted in an idealized or overly formal sense unless specifically defined as here.

[0064] Those skilled in the art of the present technology can understand that the "client", "terminal", and "terminal device" used herein include both devices with a wireless signal receiver that only has the ability to receive and does not have the ability to transmit, and devices with both receiving and transmitting hardware that can perform two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, etc., which have a single-line display or a multi-line display or a cellular or other communication device without a multi-line display; PCS (Personal Communications Service), which can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, a pager, Internet / intranet access, a web browser, a notepad, a calendar, and / or a GPS (Global Positioning System) receiver; conventional laptop and / or palm computers or other devices, which are conventional laptop and / or palm computers or other devices with and / or including a radio frequency receiver. The "client", "terminal", and "terminal device" used herein can be portable, transportable, installed in a vehicle (air, sea, and / or land), or suitable for and / or configured to run locally, and / or run in a distributed form at any other location on the earth and / or in space. The "client", "terminal", and "terminal device" used herein can also be a communication terminal, an Internet access terminal, a music / video playback terminal, for example, it can be a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback function, or can also be a smart TV, a set-top box, etc.

[0065] The hardware referred to by names such as "server", "client", and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer, and is a hardware device with the necessary components disclosed by the von Neumann principle, including a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. The computer program is stored in its memory, and the central processing unit loads the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input / output devices to complete specific functions.

[0066] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0067] Some technical solutions of this application can be deployed on a cloud server, which can realize data communication connection with business-related servers to coordinate online services, and can also form a logically associated service cluster with other related servers to provide services for related terminal devices such as smart phones, personal computers, third-party servers, etc. The smart phones and personal computers can access the Internet through well-known network access methods, and establish data communication links with the servers of this application to access and use the services provided by the servers.

[0068] For the server, generally, a service engine that provides online services opens a corresponding program interface for remote calls by various terminal devices. The relevant technical solutions suitable for deployment on the server in this application can be implemented in the server in this way.

[0069] The computer program referred to in this application, i.e., the application, is developed in a computer programming language and installed in a computer device, including a server, a terminal device, etc., and is used to implement the relevant functions specified in this application. Unless otherwise specified, it has nothing to do with the development language used.

[0070] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0071] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0072] See also Figure 1The following is a schematic diagram of the basic process of the classification model update method of the present application in its typical embodiment. A classification model update method provided by the present application is programmed as an application program and deployed to run in a computer device, and it includes the following steps:

[0073] Step S1100: Obtain a reference text that meets the preset timeliness, and perform word segmentation on it to extract candidate words:

[0074] The source of the reference text can be determined in various ways. For example, it can be a data source formed after offline manual collection, or a data source composed of website page information obtained by web crawling technology online, etc. The data source mainly provides text information. The content of this text information is usually the latest information in the region where the Internet business platform applying the present application is located, including but not limited to the latest laws and regulations, the latest national policies, the latest cultural security information, the latest current affairs hot news, etc. Summarizing the characteristics of this text information is that it needs to meet a certain preset timeliness, such as the last three days, the last week, the last month, etc. In order to meet this preset timeliness, when obtaining the text information for extracting the reference text from the data source, the time field of the relevant text information can be restricted, and the data source within the limited time range can be obtained to filter out the data source that may no longer have information contribution value in terms of time. It can be seen that the determination of the data source is related to the timeliness and accuracy of the subsequent new word discovery in the present application. However, those skilled in the art can flexibly determine the data source required for implementing the present application by appropriately combining prior knowledge according to the spirit disclosed herein.

[0075] The typical embodiment of the present application is proposed to meet the needs of a classification model suitable for classifying title texts in the e-commerce field, especially in the cross-border e-commerce field. To adapt to this classification model, the reference text processed based on the text information of the data source will ultimately be processed with reference to a classification thesaurus to discover new words therein. In this embodiment, the keywords in the preset classification thesaurus are mainly used to store some prohibited words, that is, the keywords that the e-commerce platform needs to mask, filter, and delete, and these keywords are classified into different categories. In other embodiments, if the function of the classification model is for other purposes, such as for the purpose of criminal investigation clues in public security, in this case, the classification structure of its classification thesaurus may be inconsistent with the classification structure of the classification thesaurus in the typical embodiment of the present application. Only by correspondingly replacing the classification thesaurus can the corresponding purpose be achieved. It can be seen that although the present application mainly reveals various specific implementation manners by taking the application in the e-commerce field as an example, applying the technical solution of the present application to other fields does not exceed the protection scope of the present application, and readers should understand this.

[0076] After obtaining the citation text required for this application, various tokenizers well-known to those skilled in the art can be used to tokenize and extract it, so as to obtain candidate words in the citation text through tokenization, and then determine the real central word in the citation text based on the candidate words subsequently.

[0077] Step S1200: Combine the self-information entropy and mutual information entropy of each candidate word to obtain the information score of each candidate word, and use the candidate words with information scores higher than the preset threshold as the hit words in the citation text:

[0078] In the prior art, there are various tokenizers well-known to those skilled in the art. Using different algorithms, they can be applied to tokenize and extract the citation text, including dictionary-based tokenization algorithms, statistical machine learning tokenization algorithms, neural network-based tokenizers, etc. Those skilled in the art can flexibly select a suitable tokenizer to extract candidate words from the citation text. The difference between this application and the prior art is that it performs statistics based on information entropy on the candidate words obtained by tokenization to obtain the corresponding information scores of each candidate word, and determines whether the candidate word is a truly hit new word, that is, the hit word referred to in this application, according to the information scores. For example, in one embodiment, n-gram can be used for new word discovery, the word frequencies of each candidate word are counted, and on this basis, the information score calculation based on self-information entropy and mutual information entropy is performed.

[0079] The self-information entropy of a candidate word is used to measure the degree of freedom of the candidate word. Assume that a candidate word appears N times in total, and there are n Chinese characters that have appeared on its left, and each Chinese character appears N1, N2,..., Nn times in turn, so N = N1 + N2 +... + Nn is satisfied. Therefore, the probabilities of the Chinese characters appearing on the left side of the word can be calculated, and the left neighbor entropy can be calculated according to the entropy formula. Similarly, the right neighbor entropy can be calculated. According to the sum of the left neighbor entropy and the right neighbor entropy, the self-information entropy of the candidate word can be further determined. The smaller the entropy, the lower the degree of freedom, otherwise the higher the degree of freedom.

[0080] The mutual information entropy of a candidate word is used to measure the degree of coagulation of the candidate word. Assume that the candidate word is S. First, calculate the probability P(S) of the word appearing in the candidate word set, and then try all possible binary segmentations of S, that is, divide it into the left half part sl and the right half part sr and calculate P(sl) and P(sr). For example, a two-Chinese-character word has one binary segmentation, and a three-Chinese-character word has two binary segmentations. Next, calculate the minimum value of P(S) / (P(sl)×P(sr)) in all binary segmentation schemes, and take the logarithm to be used as the measure of the degree of coagulation. The larger the mutual information entropy, the higher the degree of aggregation inside the candidate word, and the larger the mutual information entropy, the lower the degree of aggregation inside the candidate word.

[0081] The self-information entropy of the candidate word is fused to determine the information score of the candidate word, which is essentially to give the value evaluation index of the candidate word. When the information score is higher, the possibility of constituting a central word with an independent status is higher. Otherwise, when the information score is lower, the possibility of constituting a central word is lower. The simplest fusion method can be to directly add the mutual information entropy and the self-information entropy to determine. Other embodiments of this application will provide other methods for achieving fusion, which will not be shown here for the time being.

[0082] By examining the characteristics of the title text in the e-commerce field where this application is mainly used, it can be known that the title text is in the form of a title for an e-commerce product, including multiple semantically related and grammatically discrete words used to describe the e-commerce product, such as "getting together, disco, dance shoes, best-selling, new models, profitable and customized models", where, after being segmented, multiple candidate words will be obtained in theory, such as: "getting together", "disco", "dance shoes", "best-selling", "new models", "profitable", "customized models", and by examining the characteristics of these candidate words, it can be seen that they are basically stacked keywords, and there is a certain semantic correlation between words, but they are relatively discrete in grammar, and are more parallel, and the length of the entire title text will basically not be too long. According to these characteristics, considering that the mutual information entropy will be affected by the length of the candidate words, the longer the candidate words, the larger the value of the mutual information entropy, therefore, when calculating the mutual information entropy, the n-side weight can be introduced for the calculation of the mutual information entropy, where n refers to the number of characters in the candidate word.

[0083] After obtaining the information score of each candidate word by calculation, it can be determined whether the information score of the candidate word meets the requirements according to the preset threshold, and the candidate word that meets the requirements is determined as the hit word required by this application, that is, the new word determined after discovery through data mining. The preset threshold can be determined according to the prior knowledge of those skilled in the art, or it can be determined based on the statistical mean of the information scores of all candidate words. After the preset threshold is given, the information score of each candidate word is compared with it, and the candidate word with an information threshold higher than the preset threshold can be determined as the hit word found in the reference text, otherwise it is not determined as the hit word.

[0084] Step S1300: Calculate the matching degree between each hit word and each category in the preset classification word library, and cluster each hit word into each category according to the corresponding matching degree to form the keyword of the classification word library:

[0085] The classification model of the present application is adapted to the needs of realizing its title text classification function, and references a classification vocabulary library so as to realize the classification of the title text by referencing the data of the classification vocabulary library. A classification structure is established in the classification vocabulary library, and the classification structure includes multiple categories for storing a large number of keywords. Each keyword is annotated with the matching data when it is mapped to each category to characterize the similarity of a keyword belonging to a specific category.

[0086] In this embodiment, for each hit word, its matching degree with respect to each category in the classification thesaurus can be calculated to obtain respective matching degree data when the hit word is mapped to each category. This calculation process can be implemented by means of a classifier commonly used in the art, so as to obtain the corresponding matching degree of a hit word with respect to each category.

[0087] After determining the matching degree data of each hit word, each hit word can be stored in the said classification thesaurus. Correspondingly, the matching degree data mapped by each hit word to each category is also filled in, so that the hit word constitutes a keyword in the said classification thesaurus.

[0088] After storing all the hit words in the said classification thesaurus, the clustering of all the hit words discovered by the new word discovery is completed.

[0089] It can be seen that in this process, the matching degree data of the hit word plays the role of a label for the cited text. Among them, the category with the highest matching degree of the hit word can be regarded as the annotation information affecting the information value classification of the cited text. When it participates in the training in the classification model, this annotation information actually plays the role of a supervision label and can be used to supervise the learning result of the classification model for the said cited text.

[0090] Step S1400: Determine the category to which each hit word belongs as the category with the highest matching degree of the hit word, and update the mapping relationship between the hit word, its category to which it belongs, and the said cited text to the corpus:

[0091] For the convenience of the classification model of the present application to be trained according to the hit words obtained by the new word discovery, the category with the highest matching degree of each hit word is determined as the category to which the hit word belongs. Then, the mapping relationship data between the said cited text and the category to which the hit word in it belongs is updated to the corpus required for the training of the classification model, so as to add new training corpus to the said corpus, so that the said classification model can continuously improve its classification and recognition ability for the title text on the basis of the newly added corpus.

[0092] Step S1500: Start the training of the classification model for realizing the classification of the title text, so that the classification model uses the said cited text in the corpus as the training sample, and uses the mapping relationship between the hit word corresponding to the said cited text in the corpus and its category as the supervision label of the said cited text:

[0093] The corpus described above can be understood to store the training samples and supervisory labels required for the training of the classification model, wherein the reference text of the present application is added to the corpus as a training sample, and the mapping relationship data between the hit word contained in the reference text and the classification to which it belongs is added to the corpus to constitute the supervisory label required for the reference text to be used for training. The classification model classifies and identifies the reference text, and obtains the score values ​​of each classification of the classification result mapped to the classification vocabulary, wherein the classification with the highest score value is the specific classification identified according to the classification of the reference text. Whether this specific classification is correct can be compared by citing the corresponding supervisory label. If the classification label is incorrect, the correct result is approximated by correcting the learning parameters of the classification model. By citing multiple training samples to perform multiple iterative training on the classification model, the accuracy of its judgment results can be continuously improved, and its ability to classify new words can be continuously compatible and expanded.

[0094] The classification model can be any of the well-known classification models suitable for implementing security attributes in the art, including traditional machine learning models, deep learning models based on neural networks, etc. Those skilled in the art will understand that the classification model will include feature processing components such as Bert, ALBert, etc. that can vectorize the input title text, as well as classifiers suitable for performing classification operations such as Softmax classifiers, etc. Preferably, the classification model can be constructed based on the attention mechanism. Those skilled in the art will understand that the classification model of the present application does not need to be enumerated in detail, because it covers any classification model suitable for using the corpus and classification vocabulary prepared by the present application for representation learning and realizing the classification recognition function, as long as it can learn the classification ability of the title text according to the corpus and classification vocabulary of the present application, it does not exceed the protection scope of the present application.

[0095] In this embodiment, the classification vocabulary can be classified based on the security attributes of the keywords therein to form a classification structure, for example, including specific classifications such as "pornography", "gambling", and "drugs". When the classification model of the present application is used in the e-commerce field to identify e-commerce titles, with the help of this classification vocabulary, the classification model can identify whether the title text belongs to any of the categories, thereby realizing the identification of the security attributes of the title text.

[0096] In view of the particularity of the content of e-commerce titles used to describe e-commerce products in terms of grammar and semantics, and to meet the needs of a classification model for classifying and discriminating title texts of such titles, the present application improves the update process of such a classification model. Specifically, it realizes the automatic acquisition and automatic update of the content of the corpus and classification thesaurus required by such a classification model, so that the corpus and classification thesaurus of such a classification model can automatically obtain reference texts from external public resources for a long time, discover new words from them, give classification information related to the new words, fill the classification information of the new words and their reference texts into the corpus required by the classification model, and drive the updated corpus of the classification model to perform model update training, so that the classification model continuously expands its new word recognition ability and can adapt to information security specifications related to different countries, different cultures, and different current affairs hotspots, thereby ensuring that the classification model can timely identify target words such as sensitive words and violation words in e-commerce titles, and continuously and dynamically improve the information security recognition ability of e-commerce platforms.

[0097] Since the data sources for new word discovery in the present application can be obtained in various forms, especially by obtaining reference texts that meet the timeliness requirements from the public network, these reference texts can timely reflect sensitive information related to current affairs hotspots, current policies, current laws and regulations, etc. Therefore, it is ensured that the newly discovered words (hit words) are valuable for monitoring security information related to social hotspots. Further, since the self-information entropy and mutual information entropy of candidate words are integrated to examine the information value of candidate words during new word mining, the mined hit words have a high accuracy rate. Therefore, it is comprehensively ensured that hit words related to social hotspots can be timely and accurately extracted to enhance the information security risk control ability of the classification model of the present application.

[0098] Furthermore, applying the technical solution of the present application to the identification of illegal products and illegal products in the cross-border e-commerce field can continuously and rapidly improve the pre-training corpus of the relevant classification model through new word discovery according to the characteristics of different countries and different cultures, quickly update its samples, timely iterate the classification model, quickly shorten the update time of the classification model, greatly improve the detection accuracy rate of potential risk information, and minimize the human time cost required for labeling training samples.

[0099] Please refer to Figure 2 , in the embodiment provided to improve the efficiency of data source acquisition, the step S1100 includes the following steps:

[0100] Step S1111, call the crawler interface to crawl the website information page belonging to the target geographical area and within the preset time range:

[0101] In order to prepare the quoted text, taking into account the timeliness and the geographical area that the e-commerce platform targets, relevant website information pages can be obtained from the public network in a targeted manner to extract the text information therein for preparing the quoted text.

[0102] The crawler interface provided by the crawler tool can be used to crawl the website information page. The crawled website information page is generally subject to some restrictions. For example, the country to which the website's IP address belongs can be used to determine whether it belongs to the target country served by the e-commerce platform, so that only website information pages related to the target country are crawled; the information release time of the website information page can be used to determine whether it meets the preset timeliness, so that only website information pages released within a period of time are crawled.

[0103] In order to obtain the website information page of the target country, constraints on the target geographical area can be given. In order to obtain the website information page that meets the preset timeliness, constraints on the target time range can be given. The two are combined to filter the captured website information pages, so that the website information pages that meet the requirements can be obtained.

[0104] Preferably, these website information pages may be relevant pages of information such as laws, regulations and policies on government websites, relevant pages of news released by media organizations, or other similar pages.

[0105] Step S1112: extract text information from the website information page and format and pre-process it into the quoted text:

[0106] Since there are various redundant information in the website information page, only the text information of the main text part can be extracted here, and various well-known formatting preprocessing tools can be used to format and preprocess it, and finally form the cited text.

[0107] Step S1113: calling a word segmenter to segment the reference text to obtain the candidate words:

[0108] As mentioned above, various well-known word segmenters suitable for the subsequent information entropy statistics of candidate words in this application can be called here to segment the quoted text to extract the candidate words in the quoted text.

[0109] In this embodiment, by standardizing the data sources of the cited texts, the data sources are restricted to various website information pages of a classification nature that are dynamically published on the public network and are helpful for discovering the classification thesaurus that meets the present application. Since such data sources can be quickly obtained, processed, and prepared by crawler tools and have the characteristic of quickly reflecting social dynamics, the cited texts obtained thereby, after being used for new word discovery, can enrich the corpus and classification thesaurus required by the classification model, and obviously can more quickly improve the ability of the classification model to complete classification based on new words.

[0110] Please refer to Figure 3 , in the specific embodiment provided for facilitating programming implementation, the step S1200 includes the following steps:

[0111] Step S1211: Calculate the self-information entropy of each candidate word. The self-information entropy is the sum of the left neighbor information entropy and the right neighbor information entropy of the candidate word:

[0112] First of all, entropy represents the mathematical expectation of the self-information of a random variable X with a distribution of P(xi). Its calculation formula is a well-known formula in information theory and is expressed in the present application as:

[0113] H(x) = ∑P xi LnP xi

[0114] In this step, first calculate the self-information entropy of the candidate words extracted from the cited texts. In order to calculate the self-information entropy, it is necessary to first calculate the left and right neighbor entropies of the candidate words.

[0115] Left neighbor entropy:

[0116] EL = ∑P(wi|str)LnP(wi|str)

[0117] Where Wi is the set of left neighbor strings, str is the combination of strings, and P(.) is the conditional probability of wi under the given condition str.

[0118] Right neighbor entropy:

[0119] ER = ∑P(wi|str)LnP(wi|str)

[0120] Where wi is the set of right neighbor strings, str is the combination of strings, and P(.) is the conditional probability of wi under the given condition str.

[0121] According to the left neighbor entropy and right neighbor entropy formulas, it can be known that when the set of wi is larger and when the value of p is larger, the obtained entropy will be larger. The larger the entropy, the greater the amount of information represented.

[0122] On the basis of obtaining the left neighbor entropy and the right neighbor entropy, the following self-information entropy evaluation function is applied:

[0123] L = ER ln EL + EL ln ER + |EL - ER| ln(ER + EL) - (ER + EL) ln |ER - EL|

[0124] It can be seen that when the information entropy on both the left and right is relatively large, the greater the possibility that the candidate word at this time can be independently used as a central word, and the better the evaluation function optimized by the richness of the left and right adjacent words. In the actual test of this application, good results have also been achieved.

[0125] Step S1212: Calculate the mutual information entropy of each candidate word:

[0126] Generally, the mutual information between two discrete random variables X and Y can be defined as:

[0127]

[0128] where p(x, y) is the joint probability distribution function of X and Y, and p(x) and p(y) are the marginal probability distribution functions of X and Y respectively.

[0129] The mutual information evaluation between two strings measures their meaning together. The greater the mutual information, the stronger the mutual dependence, rather than being dispensable. Therefore, the following function is used to calculate the mutual information entropy:

[0130]

[0131] where P(str) is the text word frequency.

[0132] Furthermore, since the mutual information amount between objects is affected by the length of the candidate word (the longer the candidate word, the larger the mutual information value), in the cross-border e-commerce scenario, the text is mostly composed of keyword stacking, and the length of the text is generally not too long. Therefore, a weight of n squared is added to MI and corrected as follows:

[0133]

[0134] Step S1213: Adapt each candidate word, linearly fuse its self-information entropy and its mutual information entropy with different weights to obtain the information score of the corresponding candidate word:

[0135] In order to obtain the sum value between the self-information entropy and the mutual information entropy of each candidate word and facilitate their reconciliation, different weight parameters are respectively assigned to the self-information entropy and the mutual information entropy for calculating the information score. Those skilled in the art can flexibly determine the different weights according to the actual situation. The corresponding formula is as follows:

[0136] score = α·L(W) + β·AMI(W)

[0137] Among them, L(W) is the self-information entropy, AMI(W) is the mutual information entropy, and α and β are their corresponding weight parameters.

[0138] According to this formula, the comprehensive information entropy corresponding to each candidate word can be obtained as its corresponding information score, and thus the candidate word can be selected accordingly.

[0139] Step S1214: Output the candidate words with information scores higher than the preset threshold as the hit words in the reference text.

[0140] In order to achieve the optimization of candidate words, a preset threshold can be introduced to judge whether the information score of each candidate word meets the requirements. When the information score corresponding to a candidate word is higher than the preset threshold, it is determined as the hit word in the reference text where the candidate word is located.

[0141] In the process of optimizing candidate words to identify hit words in this embodiment, the thinking of weighted fusion of self-information entropy and mutual information entropy in information theory is applied. It not only considers the stability of the aggregation degree within the candidate word itself but also considers the association freedom between the candidate word and other words, making the newly discovered words more semantically accurate and more helpful for enhancing the recognition and classification ability of the classification model of this application.

[0142] Please refer to Figure 4 , in the specific embodiment provided for facilitating programming implementation, the step S1300 includes the following steps:

[0143] Step S1311: Vectorize the hit words and the keywords in the preset classification thesaurus.

[0144] As mentioned above, a feature extractor well-known to those skilled in the art for implementing word embedding can be used to vectorize the hit words discovered by the new word discovery respectively to obtain word vectors. At the same time, the keywords in the classification thesaurus of this application have also been pre-converted into word vectors by this feature extractor.

[0145] Step S1312: According to the word vectors obtained by vectorization, calculate the average cosine similarity between each hit word and the keywords of each classification in the classification thesaurus as the matching degree.

[0146] Based on the word vectors, the cosine similarity algorithm well-known to those skilled in the art can be used to calculate the cosine similarity between each hit word and the keyword of each classification in the classification thesaurus. The cosine similarities of the same classification are averaged to obtain the corresponding average cosine similarity of this classification. Thus, for a hit word, for each specific classification, an average cosine similarity is obtained as the so-called matching degree. It can be understood that the classification with the highest average cosine similarity is the classification to which the hit word most likely belongs in the classification thesaurus. Accordingly, it can also be determined that the hit word belongs to the corresponding classification.

[0147] Step S1313: Store the corresponding relationship data between each hit word and the matching degree of each classification into the classification thesaurus, so that the hit word constitutes the keyword therein, and complete the clustering of the hit word.

[0148] The matching degrees of the previously obtained hit words belonging to each specific classification constitute the corresponding relationship data, which can be stored into the classification thesaurus, so that the hit word constitutes the newly added keyword of this classification thesaurus. When all hit words are stored into the classification thesaurus, the clustering of all hit words is completed.

[0149] In this embodiment, the cosine similarity is specifically used to classify the hit words. Its implementation is simple and the running efficiency is high, which helps to improve the running efficiency of the application program implementing the technical solution of this application.

[0150] Please refer to Figure 5 , in the embodiment of further implementing the self-training and extended solution of this application based on the machine learning principle, in the step S1300, after the step S1313, the following steps are further included:

[0151] Step S1314: Call the classification model for implementing the title text classification to classify the reference text, and determine the specific classification to which the reference text belongs in the classification structure of the preset classification thesaurus.

[0152] As described above, the classification model of this application is trained by the training samples of the corpus of this application and can use the classification thesaurus to realize the classification and recognition of the title text. Therefore, in this embodiment, conversely, this classification model can be used to assist in determining whether each hit word has a good classification reference effect. For this purpose, in this step, the classification model can be called to classify and recognize the reference text, and determine which specific classification the reference text belongs to in the classification structure of the preset classification thesaurus through this classification model. Since the end output of the classification model is generally the probability score output of a multi-classifier, it can be compared in the same sense with the classification matching degree of the hit words stored into the classification thesaurus.

[0153] Step S1315: Compare whether the specific classification of the cited text is consistent with the classification of the most highly matched term in the cited text. When the two are inconsistent, recall the corresponding terms from the preset classification thesaurus:

[0154] The specific classification determined by the classification model for the cited text can be compared with the classification (with the highest matching degree) to which the term corresponding to the cited text belongs. If the two are inconsistent, at this time, it can be highly suspected whether the classification to which the term itself belongs is correct. Therefore, all terms with relatively large inconsistencies can be recalled.

[0155] Step S1316: Receive the re-annotated data for the matching degree of the recalled terms and store it in the preset classification thesaurus:

[0156] For the recalled terms, manual annotation can be performed to form effective supervision labels. This step is responsible for receiving the re-annotated data given for the recalled terms, that is, specifying their specific belonging classification or giving the matching degree data for each classification. On this basis, the terms with completed re-annotation are stored in the preset classification thesaurus to further improve the classification accuracy of the terms.

[0157] Of course, those skilled in the art can also perform brute-force recall on the keywords in the classification thesaurus, mainly the recalled terms, based on keyword queries and then perform re-annotation. However, its efficiency is obviously lower. And this embodiment expects to construct a self-training mechanism for the recalled terms, so that the recalled terms obtained according to the present application can more efficiently identify the terms that need to be recalled and revised with the help of the reverse inspection of the classification model, thereby further improving the processing efficiency.

[0158] Please refer to Figure 6 , in an embodiment further extended for the specific application of the classification model of the present application in the e-commerce field, after the classification model is trained to the convergence state and starts running, it includes the following steps:

[0159] Step S2100: Obtain the externally input title text:

[0160] When calling the classification model of the present application that has been pre-trained to the convergence state and put on the line for operation, the user will input one or more title texts to it. In the e-commerce field, the title text is generally a title text or its brief description text used to describe various attributes of e-commerce products. The classification model of the present application can classify each title text.

[0161] Step S2200: Segment and vectorize the title text to obtain the title word vector:

[0162] Similarly, the title text, as natural language, needs to be vectorized to obtain the corresponding title word vectors through word embedding, so as to perform subsequent operations for classification on this basis.

[0163] Step S2300: Classify the title text according to the title word vectors to obtain the scoring values for each classification in the classification thesaurus to which it belongs.

[0164] After the classification model of this embodiment recommends vectorization based on ALBert, operations based on Q, K, and V can be performed based on the Attention mechanism, so that when classifying the title text, the reference to the classification thesaurus is realized, and thus the scoring values for each classification in the classification structure of the classification thesaurus corresponding to the title text are obtained.

[0165] Step S2400: Output the classification label of the classification with the largest scoring value.

[0166] It can be understood that among the scoring values of each classification obtained from the classification result, the classification with the largest scoring value is the specific classification to which the title text should be assigned. Therefore, this specific classification label can be output to call and return the corresponding classification result data to the user.

[0167] This embodiment reveals that the classification model trained with the corpus provided by this application can quickly and timely learn new words, and classify title texts such as e-commerce titles with high accuracy according to the new words, which is especially suitable for cross-border e-commerce application scenarios.

[0168] Please refer to Figure 7, the embodiment of the present application further provides a classification model updating device, which includes a text citation module 1100, a text scoring module 1200, a classification updating module 1300, a corpus updating module 1400, and a model starting module 1500. Among them, the text citation module 1100 is used to obtain citation texts that meet the preset timeliness, and perform word segmentation on them to extract candidate words; the text scoring module 1200 is used to fuse the self-information entropy and mutual information entropy of each candidate word to obtain the information score of each candidate word, and use the candidate words with information scores higher than the preset threshold as the hit words in the citation text; the classification updating module 1300 is used to calculate the matching degree of each hit word with each classification in the preset classification thesaurus, and cluster each hit word into the respective classifications according to the corresponding matching degree to form the keywords of the classification thesaurus; the corpus updating module 1400 is used to determine the classification with the highest matching degree of each hit word as the classification to which the hit word belongs, and update the mapping relationship between the hit word, its classification, and the citation text to the corpus; the model starting module 1500 is used to start the training of the classification model for realizing title text classification, so that the classification model uses the citation text in the corpus as the training sample, and uses the mapping relationship between the hit word corresponding to the citation text in the corpus and its classification as the supervision label of the citation text.

[0169] In a preferred embodiment, the text citation module 1100 includes: a crawler scraping sub-module, which is used to call the crawler interface to scrape the website information pages belonging to the target geographical area and within the preset time range; a text extraction sub-module, which is used to extract the text information from the website information pages, and format and preprocess it into the citation text; a text word segmentation sub-module, which is used to call the word segmenter to perform word segmentation on the citation text to obtain the candidate words.

[0170] In a specific embodiment, the text scoring module 1200 includes: a self-information entropy sub-module, which is used to count the self-information entropy of each candidate word, and the self-information entropy is the sum value of the left adjacent information entropy and the right adjacent information entropy of the candidate word; a mutual information entropy sub-module, which is used to count the mutual information entropy of each candidate word; a score fusion sub-module, which is used to adapt to each candidate word, linearly fuse its self-information entropy and its mutual information entropy with different weights to obtain the information score of the corresponding candidate word; a word selection bubbling sub-module, which is used to output the candidate words with information scores higher than the preset threshold as the hit words in the citation text.

[0171] In a specific embodiment, the classification update module 1300 includes: a vector mapping sub-module, configured to vectorize the hit words and the keywords in the preset classification thesaurus; a matching calculation sub-module, configured to calculate the average cosine similarity between each hit word and the keywords of each classification in the classification thesaurus as the matching degree according to the word vectors obtained by vectorization; a clustering update sub-module, configured to store the corresponding relationship data between each hit word and the matching degrees of each classification in the classification thesaurus, so that the hit word constitutes a keyword therein, and complete the clustering of the hit word.

[0172] In a further extended embodiment, the classification update module 1400 further includes: a recall classification sub-module, configured to call the classification model for implementing title text classification to classify the reference text, and determine the specific classification to which the reference text belongs in the classification structure of the preset classification thesaurus; a comparison recall sub-module, configured to compare whether the specific classification of the reference text is consistent with the classification with the highest matching degree of the hit words of the reference text, and when the two are inconsistent, recall the corresponding hit words from the preset classification thesaurus; a label refreshing sub-module, configured to receive the re-annotation data for the matching degrees of the recalled hit words and store them in the preset classification thesaurus.

[0173] In a further extended embodiment, the classification model includes: a title acquisition sub-module, configured to acquire an externally input title text; a title vector sub-module, configured to segment and vectorize the title text to obtain title word vectors; a title scoring sub-module, configured to classify the title text according to the title word vectors to obtain the scoring values for each classification in the classification thesaurus; a title classification sub-module, configured to output the classification label of the classification with the largest scoring value.

[0174] In a preferred embodiment, the title text is in the form of a title of an e-commerce product, including a plurality of semantically related but grammatically discrete words for describing the e-commerce product.

[0175] The embodiments of the present application further provide a computer device. Specifically, please refer to Figure 8 , Figure 8 which is the basic structure block diagram of the computer device in this embodiment.

[0176] As Figure 8As shown, it is a schematic internal structure diagram of a computer device. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected through a system bus. Among them, the non-volatile storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database can store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a classification model update method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device can store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute a classification model update method. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Figure 7 The structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0177] In this embodiment, the processor is used to execute Figure 7 the specific functions of each module / sub-module in, and the memory stores the program codes and various types of data required to execute the above modules. The network interface is used for data transmission between the user terminal and the server. The memory in this embodiment stores the program codes and data required to execute all sub-modules in the classification model update device, and the server can call the program codes and data of the server to execute the functions of all sub-modules.

[0178] This application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the steps of the classification model update method in any of the above embodiments.

[0179] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), etc., or a random access memory (RAM), etc.

[0180] It should be understood that the classification model of the technical solution of the present application is applicable not only to the security classification and recognition of e-commerce titles, but also to the classification and recognition of statements in application scenarios with the same characteristics as e-commerce titles. In this regard, those skilled in the art should be aware that applying the technical solution of the present application to other scenarios outside the e-commerce field still falls within the scope of equivalent replacement defined by the present application.

[0181] In summary, the present application can provide timely updates to the corpus and classification thesaurus for the classification model for realizing title text classification, enabling the classification model to continuously introduce its semantic understanding ability of dynamically newly added hot information after iterative training, so as to more efficiently, accurately and timely classify the security attributes of title text.

[0182] Those skilled in the art of the present technology can understand that the steps, measures, and solutions in the various operations, methods, and processes discussed in the present application can be alternated, changed, combined, or deleted. Further, the other steps, measures, and solutions in the various operations, methods, and processes discussed in the present application can also be alternated, changed, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and solutions in the prior art that are the same as those disclosed in the various operations, methods, and processes in the present application can also be alternated, changed, rearranged, decomposed, combined, or deleted.

[0183] The above are only partial embodiments of the present application. It should be noted that for those of ordinary skill in the art of the present technology, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.

Claims

1. A method for updating a classification model, characterized in that Applied to an e-commerce platform, it includes the following steps: Obtain a reference text that meets the preset timeliness, perform word segmentation on it to extract candidate words. The reference text is sourced from website information pages within the target geographical area and a preset time range, and contains the latest information of the target geographical area where the e-commerce platform is located; Fuse the self-information entropy and mutual-information entropy of each candidate word to obtain the information score of each candidate word, and use the candidate words with information scores higher than the preset threshold as the hit words in the reference text; Calculate the matching degree of each hit word with each category in the preset classification thesaurus, obtain the respective matching degrees when the hit word is mapped to each category, and store the matching degrees corresponding to each category mapped by each hit word in the classification thesaurus to achieve clustering, so that the hit word constitutes the keyword of the classification thesaurus. The classification thesaurus is classified based on the security attributes of the keywords therein; Determine the category with the highest matching degree of each hit word as the category to which the hit word belongs, and update the mapping relationship between the hit word, its category, and the reference text to the corpus; Start the training of the security attribute classification model for realizing title text classification, so that the security attribute classification model uses the reference text in the corpus as the training sample, and uses the mapping relationship between the hit word corresponding to the reference text in the corpus and its category as the supervision label of the reference text. After the training is completed, start the security attribute classification model for security identification of the title text of e-commerce products corresponding to the target geographical area. After the security attribute classification model is trained to the convergence state and starts running, it includes the following steps: Obtain the externally input title text; Segment and vectorize the title text to obtain the title word vector; Classify the title text according to the title word vector to obtain the scoring values belonging to each category in the classification thesaurus; Output the classification label of the category with the largest scoring value, where the title text is in the title form of an e-commerce product and includes multiple semantically related but grammatically discrete words for describing the e-commerce product.

2. The classification model updating method according to claim 1, wherein Obtain a reference text that meets the preset timeliness, perform word segmentation on it to extract candidate words, including the following steps: Call the crawler interface to crawl the website information pages within the target geographical area and the preset time range; Extract the text information therein from the website information pages, and format and preprocess it into the reference text; Call the word segmenter to perform word segmentation on the reference text to obtain the candidate words.

3. The classification model update method according to claim 1, wherein Fuse the self-information entropy and mutual-information entropy of each candidate word to obtain the information score of each candidate word, and use the candidate words with information scores higher than the preset threshold as the hit words in the reference text, including the following steps: Statistical the self-information entropy of each candidate word, and the self-information entropy is the sum value calculated according to the left adjacent information entropy and the right adjacent information entropy of the candidate word; Statistical the mutual-information entropy of each candidate word; Adapt each candidate word, linearly fuse its self-information entropy and its mutual-information entropy with different weights to obtain the information score of the corresponding candidate word; Output the candidate words with information scores higher than the preset threshold as the hit words in the reference text.

4. The classification model updating method according to claim 1, wherein Calculate the matching degrees of each hit word with each category in the preset classification thesaurus, obtain the respective matching degrees when the hit word is mapped to each category, and store the matching degrees corresponding to each hit word mapped to each category in the classification thesaurus to achieve clustering, so that the hit word constitutes the keyword of the classification thesaurus, including the following steps: Vectorize the hit word and the keywords in the preset classification thesaurus; According to the word vectors obtained by vectorization, calculate the average cosine similarity between each hit word and the keywords of each category in the classification thesaurus as the matching degree; Store the corresponding relationship data of the matching degrees of each hit word with the respective categories in the classification thesaurus, so that the hit word constitutes the keyword therein, and complete the clustering of the hit word.

5. The classification model update method according to claim 4, wherein Storing the matching degrees corresponding to each hit word mapped to each category in the classification thesaurus to achieve clustering further includes the following steps: Call the security attribute classification model for realizing title text classification to classify the cited text, and determine the specific category to which the cited text belongs in the classification structure of the preset classification thesaurus; Compare whether the specific category of the cited text is consistent with the category with the highest matching degree of the hit words of the cited text. When the two are inconsistent, recall the corresponding hit words from the preset classification thesaurus; Receive the re-annotation data of the matching degrees of the recalled hit words, and store them in the preset classification thesaurus.

6. A classification model updating device, characterized in that Applied in an e-commerce platform, it includes: A text citation module, configured to obtain a cited text that meets the preset timeliness, perform word segmentation on it to extract candidate words, where the cited text is from the website information page within the target geographical area and the preset time range, and contains the latest information of the target geographical area where the e-commerce platform is located; A text scoring module, configured to fuse the self-information entropy and mutual information entropy of each candidate word to obtain the information score of each candidate word, and use the candidate words with information scores higher than the preset threshold as the hit words in the cited text; A classification update module, configured to calculate the matching degrees of each hit word with each category in the preset classification thesaurus, obtain the respective matching degrees when the hit word is mapped to each category, store the matching degrees corresponding to each hit word mapped to each category in the classification thesaurus to achieve clustering, so that the hit word constitutes the keyword of the classification thesaurus, and the classification thesaurus is classified based on the security attributes of the keywords therein; A corpus update module, configured to determine the category with the highest matching degree of each hit word as the category to which the hit word belongs, and update the mapping relationship between the hit word, its category to which it belongs, and the cited text to the corpus; A model startup module, which is used to start the training of a security attribute classification model for realizing title text classification, enabling the security attribute classification model to use the reference text in the corpus as training samples, and using the mapping relationship between the hit words corresponding to the reference text in the corpus and their classifications as the supervision labels of the reference text. After the training is completed, the security attribute classification model is started to identify the security of the title text of e-commerce products corresponding to the target geographical area. After the security attribute classification model is trained to the convergence state and starts running, it includes the following steps: obtaining the externally input title text; segmenting and vectorizing the title text to obtain title word vectors; classifying the title text according to the title word vectors to obtain the scoring values of each classification in the classification thesaurus; outputting the classification label of the classification with the largest scoring value, where the title text is in the title form of an e-commerce product and includes multiple semantically related but grammatically discrete words for describing the e-commerce product.

7. The classification model updating device according to claim 6, characterized in that, The text reference module includes: A crawler scraping sub-module, which is used to call the crawler interface to scrape the website information pages belonging to the target geographical area and within the preset time range; A text extraction sub-module, which is used to extract the text information from the website information pages and preprocess it in a formatted manner into the reference text; A text segmentation sub-module, which is used to call a word segmenter to segment the reference text to obtain the candidate words.

8. The classification model updating device according to claim 6, wherein The text scoring module includes: An information entropy sub-module, which calculates the self-information entropy of each candidate word, and the self-information entropy is the sum value calculated according to the left adjacent information entropy and the right adjacent information entropy of the candidate word; A mutual information entropy sub-module, which is used to calculate the mutual information entropy of each candidate word; A score fusion sub-module, which is used to adapt each candidate word, linearly fuse its self-information entropy and its mutual information entropy with different weights to obtain the information score of the corresponding candidate word; A word selection bubbling sub-module, which is used to output the candidate words with information scores higher than the preset threshold as the hit words in the reference text.

9. A computer device, characterized in that, It includes a memory and a processor. When the computer-readable instructions stored in the memory are executed by the processor, the processor executes the steps of the classification model update method according to any one of claims 1 to 5.

10. A storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, one or more processors execute the steps of the method for updating the classification model according to any one of claims 1 to 5.

Images