Text recommendation method and related equipment based on artificial intelligence
Through artificial intelligence-based methods, automatic word segmentation, encoding and classification, and calculation of associated indicator recommendation text, the problems of high labor costs and low accuracy in the existing technology are solved, and efficient and accurate text recommendation is achieved.
Patent Information
- Application Number
- CN202210507428.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-10
- Publication Date
- 2025-04-11
- Estimated Expiration
- 2042-05-10
AI Technical Summary
The prior art relies on manual addition of category tags in text recommendations, resulting in high labor costs and low accuracy.
Using an artificial intelligence-based method, text is automatically recommended through word segmentation, encoding, multiple classifications and correlation indicator calculations. The specific steps include: using word segmentation tools to obtain vocabulary, encode and classify to obtain semantic groups, calculate the similarity of the vocabulary to be evaluated to determine the target group, and secondaryly classify the encoded data in the target group to obtain candidate groups, and finally recommend vocabulary based on the associated indicators.
Improves the accuracy of text recommendations, reduces labor costs, and improves the efficiency and meticulousness of recommendations.
Smart Images

Figure CN114818686B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based text recommendation method, device, electronic device and storage medium. Background Art
[0002] With the rapid development of information technology and the Internet, people have more and more ways to acquire knowledge in their daily lives. In recent years, the demand for searching relevant text content through the Internet and keywords has been increasing.
[0003] Currently, text recommendation is usually performed by adding category labels to texts in advance and retrieving related texts based on the labels. However, this method requires a lot of manpower to preprocess the text data, has high labor costs, and has low accuracy. Summary of the invention
[0004] In view of the above, it is necessary to provide an artificial intelligence-based text recommendation method and related equipment to solve the technical problem of how to improve the accuracy of text recommendation, wherein the related equipment includes an artificial intelligence-based text recommendation device, an electronic device and a storage medium.
[0005] The present application embodiment provides a text recommendation method based on artificial intelligence, the method comprising:
[0006] Use a preset word segmentation tool to segment the text in the natural language database to obtain multiple words;
[0007] Encoding the words to obtain encoding data corresponding to each word;
[0008] Classifying the coded data to obtain a plurality of semantic groups, each semantic group comprising a plurality of coded data;
[0009] Calculate the similarity between the encoded data corresponding to the vocabulary to be evaluated and each semantic group, and take the semantic group corresponding to the maximum similarity as the target group;
[0010] Classifying the coded data in the target group to obtain a plurality of candidate groups, each candidate group containing a plurality of coded data;
[0011] Calculating a correlation index for each candidate group, where the correlation index is used to characterize the degree to which the encoded data in each candidate group should be recommended;
[0012] The words corresponding to the coded data in the candidate group are recommended in descending order of the association index.
[0013] The above-mentioned artificial intelligence-based text recommendation method obtains the quantitative expression of the text by encoding the vocabulary, which can improve the accuracy of text recommendation, and classifies the encoded data multiple times to obtain the optimized classification results. The target group data is further screened out from the optimized classification results based on the vocabulary to be evaluated, and then multiple candidate group data are obtained by secondary classification of the target group data, and the correlation indicators of each candidate group are evaluated, and vocabulary is recommended to users based on the correlation indicators, thereby improving the accuracy of text recommendation.
[0014] In some embodiments, classifying the encoded data to obtain a plurality of semantic groups comprises:
[0015] Constructing a center list according to a preset classification upper limit, and selecting a plurality of coded data as center data according to each element in the center list;
[0016] Classifying all the coded data multiple times according to the central data to obtain multiple classification results, each classification result comprising multiple vocabulary classification groups;
[0017] Calculate the sum of squared errors of each classification result as an evaluation value, and calculate the difference of the evaluation values to construct an evaluation result hash table;
[0018] The classification result corresponding to the minimum difference in the evaluation result hash table is searched, and all vocabulary classification groups in the classification result are used as semantic groups.
[0019] In this way, multiple classification results are obtained by classifying the coded data, each classification result contains multiple coded data, and the classification results are evaluated to screen out more optimized classification results, and the vocabulary classification groups in the more optimized classification results are used as semantic groups, thereby improving the accuracy of vocabulary classification.
[0020] In some embodiments, the coded data is classified according to the central data to obtain a classification result, and the classification result comprises a plurality of vocabulary classification groups, including:
[0021] a, mark all coded data as 0 and randomly select a coded data as the current data;
[0022] b, respectively calculating the Euclidean distance between the current data and each center data;
[0023] c. Mark the current data as 1 and classify it into the category where the central data corresponding to the smaller Euclidean distance belongs;
[0024] d. Randomly select a coded data marked as 0 as the current data again, and repeat steps b to d until all coded data are marked as 1, then stop the iteration and obtain multiple vocabulary classification groups;
[0025] e. Calculate the mean of all the coded data in each vocabulary classification group. If the mean satisfies a preset discrimination condition, the vocabulary classification group is taken as the classification result. Otherwise, the coded data closest to the mean in each vocabulary classification group is taken as the central data and steps a to e are repeated to obtain the classification result.
[0026] In this way, multiple vocabulary classification groups are obtained by clustering the encoded data in an iterative manner. The vocabulary corresponding to the encoded data can be divided into multiple vocabulary classification groups with similar semantics without manual labeling of the vocabulary, thereby improving the efficiency of text recommendation.
[0027] In some embodiments, calculating the sum of squared errors of each classification result as an evaluation value, and calculating the difference of the evaluation values to construct an evaluation result hash table includes:
[0028] Calculate the error of each vocabulary classification group respectively, wherein the error is the sum of the squares of the Euclidean distances between all the encoded data and the central data in each vocabulary classification group;
[0029] Calculate the sum of errors of all vocabulary classification groups in each classification result as the evaluation value of each classification result;
[0030] The evaluation values are arranged from small to large according to the number of vocabulary classification groups in each classification result, and after calculating the difference between two adjacent evaluation values, the differences are combined according to the order of the evaluation values to construct an evaluation result hash table.
[0031] In this way, by calculating the error of each vocabulary classification group in each classification result and further obtaining the evaluation value of each classification result, an evaluation result hash table is constructed based on the evaluation value, which is convenient for subsequent screening of more optimized classification results, thereby improving the efficiency of text recommendation.
[0032] In some embodiments, classifying the encoded data in the target group to obtain a plurality of candidate groups includes:
[0033] a, classifying the coded data in the target group according to a preset radius threshold and density threshold, wherein the categories include core data and outlier data;
[0034] b. Setting a discrimination condition according to the radius threshold, wherein the discrimination condition refers to that the relationship between two data satisfies any one of the preset relationships, wherein the preset relationships include: density direct, density reachable, and density connected, wherein the density direct means that the Euclidean distance between two coded data is not less than the radius threshold; the density reachable means that the relationship between two coded data is not density direct and the two coded data have common density direct coded data; the density connected means that the relationship between two coded data is not density reachable and the two coded data have common density reachable coded data;
[0035] c, mark all core data as "not accessed";
[0036] d. Select any core data marked as “unvisited” as the target data;
[0037] e. traverse all core data labeled as "unvisited" in sequence. If the traversed core data meets the discrimination condition, the core data and the target data are classified into the same candidate group, and all core data in the candidate group are marked as "visited";
[0038] f. Repeat steps d and e to obtain multiple candidate groups, and take all outlier data as a candidate group.
[0039] In this way, multiple candidate groups are obtained by further classifying the coded data in the target group, so that the vocabulary corresponding to the coded data can be divided more finely to improve the accuracy of text recommendation.
[0040] In some embodiments, evaluating each candidate group to obtain a correlation index for each candidate group includes:
[0041] Calculate the mean of each candidate group, and take the encoded data with the smallest Euclidean distance from the mean in each candidate group as the centroid of each candidate group;
[0042] Calculate the variance of the encoded data in each candidate group as the degree of aggregation, where the degree of aggregation is used to represent the similarity of the words corresponding to the encoded data in the candidate group;
[0043] Calculating the similarity between the target data and the centroid data, wherein a higher similarity indicates that the target data is more similar to the centroid data;
[0044] The aggregation degree and the similarity are input into a custom normalization model to obtain a normalization result, and the normalization result is used as an association index for each candidate group, wherein the association index is used to characterize the degree of correlation between the encoded data in the candidate group and the target data.
[0045] In this way, by inputting the aggregation degree of the encoded data in each candidate group and the similarity between each candidate group and the target data into the customized normalization model, the corresponding association index of each candidate group is obtained, which provides data support for subsequent text recommendations, thereby improving the accuracy of text recommendations.
[0046] In some embodiments, the customized normalization model satisfies the relationship:
[0047]
[0048] Among them, T represents the association index, and the higher the value of the association index is, the more the vocabulary corresponding to the coded data in the corresponding candidate group should be recommended; x represents the aggregation degree of the candidate group, and the higher the aggregation degree is, the more important the vocabulary corresponding to the coded data in the candidate group is; y represents the similarity between the target data and the candidate group, and the higher the similarity is, the more the vocabulary corresponding to the coded data in the candidate group should be recommended; α represents a preset harmonic constant; e represents a natural constant.
[0049] In this way, the correlation index of each candidate group is obtained through the customized normalization model, which provides data support for subsequent text recommendations and can improve the accuracy of text recommendations.
[0050] The embodiment of the present application also provides a text recommendation device based on artificial intelligence, the device comprising:
[0051] A word segmentation unit uses a preset word segmentation tool to segment text in a natural language database to obtain multiple words;
[0052] An encoding unit, used for encoding the words in the natural language database to obtain encoding data corresponding to each word;
[0053] A first classification unit, configured to classify the coded data to obtain a plurality of semantic groups, each semantic group comprising a plurality of coded data;
[0054] A first calculation unit is used to calculate the similarity between the encoded data corresponding to the vocabulary to be evaluated and each semantic group, and take the semantic group corresponding to the maximum similarity as the target group;
[0055] A second classification unit, used for classifying the coded data in the target group to obtain a plurality of candidate groups, each candidate group containing a plurality of coded data;
[0056] A second calculation unit, used to calculate a relevance index for each candidate group, wherein the relevance index is used to characterize the degree to which the coded data in each candidate group should be recommended;
[0057] The recommendation unit is used to recommend the words corresponding to the coded data in the candidate group in descending order of the association index.
[0058] The present application also provides an electronic device, the device comprising:
[0059] a memory storing computer readable instructions; and
[0060] A processor executes the computer-readable instructions stored in the memory to implement the artificial intelligence-based text recommendation method.
[0061] An embodiment of the present application also provides a computer-readable storage medium, in which computer-readable instructions are stored. The computer-readable instructions are executed by a processor in an electronic device to implement the artificial intelligence-based text recommendation method.
[0062] The above-mentioned artificial intelligence-based text recommendation method obtains the quantitative expression of the text by encoding the vocabulary, which can improve the accuracy of text recommendation, and classifies the encoded data multiple times to obtain the optimized classification results. The target group data is further screened out from the optimized classification results based on the vocabulary to be evaluated, and then multiple candidate group data are obtained by secondary classification of the target group data, and the correlation indicators of each candidate group are evaluated, and vocabulary is recommended to users based on the correlation indicators, thereby improving the accuracy of text recommendation. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] Figure 1 It is a flowchart of a preferred embodiment of an artificial intelligence-based text recommendation method involved in this application.
[0064] Figure 2 It is a functional module diagram of a preferred embodiment of the artificial intelligence-based text recommendation device involved in this application.
[0065] Figure 3 It is a schematic diagram of the structure of an electronic device of a preferred embodiment of the artificial intelligence-based text recommendation method involved in the present application. DETAILED DESCRIPTION
[0066] In order to more clearly understand the purpose, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other. In the following description, many specific details are set forth to facilitate a full understanding of the present application, and the embodiments described are only a part of the embodiments of the present application, rather than all of the embodiments.
[0067] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which the present application belongs. The terms used herein in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.
[0069] The embodiment of the present application provides a text recommendation method based on artificial intelligence, which can be applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a programmable gate array (Field-Programmable Gate Array, FPGA), a digital processor (Digital Signal Processor, DSP), an embedded device, etc.
[0070] The electronic device can be any electronic product that can perform human-computer interaction with a user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (PDA), a game console, an interactive network television (IPTV), a smart wearable device, etc.
[0071] The electronic device may also include a network device and / or a user device, wherein the network device includes, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud consisting of a large number of hosts or network servers based on cloud computing.
[0072] The network where the electronic device is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a virtual private network (Virtual Private Network, VPN), etc.
[0073] like Figure 1 FIG. 1 is a flowchart of a preferred embodiment of the text recommendation method based on artificial intelligence of the present application. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.
[0074] S10, using a preset word segmentation tool to segment the text in the natural language database to obtain multiple words;
[0075] In this optional embodiment, the natural language database refers to a data set that is designed, stored, and managed according to a data structure for natural language texts. Exemplarily, the natural language database in this solution may be a MySQL database, which is an open source database whose function may be to store any natural language text.
[0076] In this optional embodiment, the preset word segmentation tool can be a jieba word segmentation tool, which is a code written by a Python program, and its function is to separate words in a natural language text to obtain multiple words. The form of the jieba word segmentation tool can be jieba.cut(sentence), where cut represents the word segmentation instruction in the jieba word segmentation tool, and sentence represents the natural language text.
[0077] Exemplarily, when the sentence is "everything looks like just waking up", the output of the jieba word segmentation tool is a list, each element in the list is a word, and the form of the list is [everything looks like just waking up].
[0078] In this way, the natural language text is preprocessed through the preset word segmentation tool to obtain multiple words, which can represent the natural language text in a more fine-grained manner and provide a basis for subsequent word encoding, thereby improving the accuracy of text recommendations.
[0079] S11, encoding the words to obtain encoding data corresponding to each word.
[0080] In this optional embodiment, each word in the natural language database can be encoded according to a preset text encoding method. The preset text encoding method can be a one-hot algorithm, which means a one-hot encoding algorithm. The function of the one-hot encoding algorithm is to encode each word in the natural language database into binary vector data. Exemplarily, the binary vector data can be in the form of [0,1,0,0,0].
[0081] In this optional embodiment, the weight corresponding to each word in the natural language database can be calculated according to a preset word weight algorithm, and the weight is used to characterize the importance of each word in the natural language database. The higher the value of the weight, the more important the corresponding word is in the natural language database. The preset word weight algorithm can be a TF-IDF algorithm. The full name of the TF-IDF algorithm is term frequency-inverse document frequency, which means term frequency-inverse document frequency algorithm. The TF-IDF algorithm can output the weight corresponding to each word, and the value of the weight can be any real number.
[0082] In this optional embodiment, the product of the weight and the binary vector data of the corresponding vocabulary can be calculated to obtain the encoding data corresponding to each vocabulary. For example, if the binary data corresponding to the vocabulary "I" is [0, 1, 0, 0, 0] and its corresponding weight is 0.23, then the encoding data corresponding to the vocabulary "I" is [0, 0.23, 0, 0, 0].
[0083] In this optional embodiment, the vocabulary may be used as a key and the encoded data corresponding to the vocabulary may be used as a value to construct a key-value pair, and all the key-value pairs may be used as an encoded hash table.
[0084] Exemplarily, when the word is "I" and its corresponding encoding data is [0, 0.23, 0, 0, 0], the key-value pair is in the form of (I, [0, 0.23, 0, 0, 0]).
[0085] In this way, the encoding data corresponding to each word is obtained by calculating the product of the importance corresponding to each word and the binary vector data corresponding to each word. The encoding data can represent the text data in a quantitative manner, thereby improving the accuracy of subsequent text recommendations.
[0086] S12, classifying the encoded data to obtain a plurality of semantic groups, each semantic group containing a plurality of encoded data.
[0087] In this optional embodiment, the coded data is classified to obtain a plurality of semantic groups, each semantic group containing a plurality of coded data, including:
[0088] Constructing a center list according to a preset classification upper limit, and selecting a plurality of coded data as center data according to each element in the center list;
[0089] Classifying all the coded data multiple times according to the central data to obtain multiple classification results, each classification result comprising multiple vocabulary classification groups;
[0090] Calculate the sum of squared errors of each classification result as an evaluation value, and calculate the difference of the evaluation values to construct an evaluation result hash table;
[0091] The classification result corresponding to the minimum difference in the evaluation result hash table is searched, and all vocabulary classification groups in the classification result are used as semantic groups.
[0092] In this optional embodiment, the central list can be constructed according to a preset classification upper limit, where the preset classification upper limit is a positive integer greater than 2, and the classification upper limit is used to control the number of times the encoded data is classified.
[0093] In this optional embodiment, each element in the central list is a positive integer, wherein each element represents the number of categories obtained after classifying the encoded data each time. Since the encoded data needs to be divided into multiple categories, the minimum value of the elements in the central list is 2.
[0094] In this optional embodiment, each positive integer greater than or equal to 2 can be taken in turn as an element in the central list, and the preset upper limit of the classification can be taken as the last element in the central list. If the preset upper limit of the classification is n, the central list has n-1 elements.
[0095] Exemplarily, when the preset upper limit of classification is 9, the central list is [2, 3, 4, 5, 6, 7, 8, 9], where the value of each element is used to represent the number of categories obtained after each classification of the encoded data.
[0096] In this optional embodiment, the elements in the central list can be selected in sequence as the number of categories, and the encoded data equal to the number of categories can be randomly selected from the encoded data as the central data. For example, if the central list is [2,3,4,5,6,7,8,9], the first number of categories selected is 2, and two encoded data can be randomly selected from all the encoded data as the central data.
[0097] In an optional embodiment, classifying all the coded data multiple times according to the central data to obtain multiple classification results includes:
[0098] a, mark all coded data as 0 and randomly select a coded data as the current data;
[0099] b, respectively calculating the Euclidean distance between the current data and each center data;
[0100] c. Mark the current data as 1 and classify it into the category where the central data corresponding to the smaller Euclidean distance belongs;
[0101] d. Randomly select a coded data marked as 0 as the current data again, and repeat steps b to d until all coded data are marked as 1, then stop the iteration and obtain multiple vocabulary classification groups;
[0102] e. Calculate the mean of all the coded data in each vocabulary classification group. If all the means meet the preset judgment conditions, the vocabulary classification group is taken as the classification result. Otherwise, the coded data closest to the mean in each vocabulary classification group is taken as the central data and steps a to e are repeated to obtain the classification result.
[0103] In this optional embodiment, the specific implementation steps of performing multiple classifications on all the encoded data to obtain multiple classification results are:
[0104] A1: Mark all coded data as 0, and randomly select a coded data as current data, where the current data corresponds to a certain word in the natural language database;
[0105] A2: Calculate the Euclidean distance between the current data and each center data respectively. For example, if the current data is [0.23, 0, 0, 0] and a center data is [0, 0.65, 0, 0], the Euclidean distance between the current data and the center data is calculated as follows:
[0106]
[0107] Then the Euclidean distance between the current data and the central data is 0.4754.
[0108] The Euclidean distance is used to represent the similarity between the current data and the central data. The smaller the Euclidean distance is, the higher the similarity between the vocabulary represented by the current data and the vocabulary represented by the central data is.
[0109] A3: Mark the current data as 1 and classify it into the vocabulary classification group where the central data corresponding to the smaller Euclidean distance belongs. For example, if there are four central data in total, and the Euclidean distance between the current data and each central data is recorded as Dis 1 =0.4754, Dis 2 =0.57, Dis 3 =0.65 and Dis 4 =0.32, where the superscript of each Euclidean distance represents the index of each center data. Since the minimum value of the four Euclidean distances is Dis 4 =0.32, so the current data is classified into the vocabulary classification group where the fourth central data is located;
[0110] A4: randomly select a coded data marked as 0 as the current data, and repeat steps A2 to A4 until all coded data are marked as 1 and multiple vocabulary classification groups are obtained;
[0111] A5: Calculate the mean of all the coded data in each vocabulary classification group. If all the means meet the preset discrimination conditions, the multiple vocabulary classification groups are taken as the classification results. Otherwise, the coded data closest to the mean in each vocabulary classification group is taken as the central data, and repeat steps A1 to A5 to obtain the classification result.
[0112] The preset judgment condition refers to that the Euclidean distance between the mean and the central data is less than a preset threshold. For example, the preset threshold may be 0.1. The purpose of this step is to ensure that the classification result changes are small enough, that is, the performance of the classification result is relatively excellent.
[0113] In this optional embodiment, each element in the central list corresponds to a classification result, each classification result corresponds to multiple vocabulary classification groups, each vocabulary classification group contains multiple encoding data, each encoding data corresponds to a vocabulary, and the encoding data in each vocabulary classification group corresponds to a class of vocabulary with the same semantics.
[0114] In an optional embodiment, the sum of squared errors of each classification result is calculated as an evaluation value, and the difference of the evaluation values is calculated to construct an evaluation result hash table, including:
[0115] Calculate the error of each vocabulary classification group respectively, wherein the error is the sum of the squares of the Euclidean distances between all the encoded data and the central data in each vocabulary classification group;
[0116] Calculate the sum of all errors in each classification result as the evaluation value of each classification result;
[0117] The evaluation values are arranged from small to large according to the number of vocabulary classification groups in each classification result, and after calculating the difference between two adjacent evaluation values, the differences are combined according to the order of the evaluation values to construct an evaluation result hash table.
[0118] In this optional embodiment, the error of each vocabulary classification group in each classification result can be calculated respectively, and the error is the sum of the squares of the Euclidean distances between all the encoded data and the central data in each vocabulary classification group. The Euclidean distance is calculated as follows:
[0119]
[0120] Where A represents the central data, B represents the coded data, and i represents the index of the dimension of the central data and the coded data. Therefore, A i represents the value of the i-th dimension of the central data, B irepresents the value of the i-th dimension of the encoded data, m represents the total number of dimensions of the central data and the encoded data, Dis represents the Euclidean distance, and the smaller the Euclidean distance is, the higher the similarity between the central data and the encoded data is, and the more similar the vocabulary represented by the central data is to the vocabulary represented by the encoded data.
[0121] For example, if the central data of a vocabulary classification group is [0, 0.23, 0, 0] and one of the coded data is [0.45, 0, 0, 0], the Euclidean distance between the central data and the coded data is calculated as follows:
[0122]
[0123] Then the Euclidean distance between the central data and the encoded data is 0.5.
[0124] For example, if a vocabulary classification group includes four coded data and one central data, and the Euclidean distances between the four coded data and the central data are 0.5, 1, 1.5 and 2 respectively, then the error corresponding to the vocabulary classification group is calculated as follows:
[0125] 0.5 2 +1 2 +1.5 2 +2 2 =7.5
[0126] The error corresponding to the vocabulary classification group is 7.5. The smaller the error is, the higher the degree of aggregation of all the coded data in the vocabulary classification group is, and the more similar the vocabularies corresponding to all the coded data in the vocabulary classification group are.
[0127] In this optional embodiment, the sum of all errors in each classification result can be calculated as the evaluation value of each classification result. The smaller the evaluation value, the better the effect of the corresponding classification result. For example, if a classification result includes four vocabulary classification groups, and the errors of the four vocabulary classification groups are 7.5, 9.5, 10.5 and 11.5 respectively, the evaluation value corresponding to the classification result is 39.
[0128] In this optional embodiment, the number of vocabulary classification groups in each classification result can be used as a key and the evaluation value corresponding to each classification result can be used as a value to construct a key-value pair, and the key-value pairs are arranged from small to large according to the value of the key.
[0129] For example, if the corresponding evaluation value is 100 when the number of vocabulary classification groups is 2, the corresponding evaluation value is 60 when the number of vocabulary classification groups is 3, and the corresponding evaluation value is 50 when the number of vocabulary classification groups is 4, then the arrangement order of the key-value pairs is (2, 100), (3, 60), and (4, 50).
[0130] In this optional embodiment, the difference between two adjacent evaluation values may be calculated. For example, if the evaluation value obtained when the number of vocabulary classification groups is 2 is 100, and the evaluation value obtained when the number of vocabulary classification groups is 3 is 60, then the difference is 40.
[0131] In this optional embodiment, the differences can be combined according to the order of the evaluation values to construct an evaluation result hash table. For example, if the evaluation value differences are 40 and 10 respectively, the evaluation result hash table is [([2,3],40),([3,4],10)].
[0132] In this optional embodiment, the key corresponding to the smallest value in the evaluation result hash table can be selected as the target key. For example, if the evaluation result hash table is [([2,3],40),([3,4],10)], the smallest value is 10 and the corresponding key is [3,4].
[0133] In this optional embodiment, the vocabulary classification group in the classification result corresponding to the smaller value in the target key can be selected as the semantic group.
[0134] For example, if the target key is [3, 4], three vocabulary classification groups in the classification result corresponding to 3 can be selected as semantic groups.
[0135] In this way, multiple classification results are obtained by classifying the encoded data multiple times, and the error of each classification result is calculated to construct an evaluation result hash table. The semantic group is obtained by querying the classification result corresponding to the minimum value in the evaluation result hash table, thereby taking into account both the efficiency and accuracy of vocabulary classification.
[0136] S13, calculating the similarity between the encoded data corresponding to the vocabulary to be evaluated and each semantic group, and taking the semantic group corresponding to the maximum similarity as the target group.
[0137] In this optional embodiment, the vocabulary to be evaluated refers to the vocabulary that needs to be associated in the natural language database. The vocabulary to be evaluated can be used as a key, and the encoding hash table can be traversed in sequence to find the value corresponding to the key as the target data.
[0138] Exemplarily, when the word to be evaluated is "description", the word "description" can be used as a key, and all keys in the encoding hash table can be traversed in turn to query the encoded data corresponding to the word "description", and the encoded data corresponding to the word "description" can be used as target data. The target data can be in the form of [0, 0.23, 0, 0, 0].
[0139] In this optional embodiment, the similarity between the target data and the central data of each semantic group may be used as the similarity between the target data and the semantic group, and the semantic group corresponding to the minimum similarity is selected as the target group.
[0140] In this optional embodiment, the Euclidean distance between the target data and the central data of each semantic group can be calculated, and the inverse of the Euclidean distance can be used as the similarity. The higher the similarity, the more similar the target data is to the central data of the semantic group, and the higher the similarity between the vocabulary represented by the target data and the vocabulary in the semantic group.
[0141] In this optional embodiment, if the target data is A and the semantic group center data is B, the similarity is calculated as follows:
[0142]
[0143] Wherein, S represents the similarity between the target data A and the semantic group central data B. The higher the value of S, the higher the similarity between the vocabulary represented by the target data A and the vocabulary represented by the central data B. i represents the dimension index of the target data and the central data, n represents the number of dimensions of the target data and the central data, and A i represents the value of the i-th dimension in the target data, B i Represents the value of the i-th dimension in the semantic group center data.
[0144] Exemplarily, when the target data is [0, 0.23, 0, 0, 0] and the semantic group center data is [0, 0.45, 0, 0, 0], the similarity is calculated as follows:
[0145]
[0146] In this optional embodiment, the semantic group corresponding to the greatest similarity can be used as the target group. For example, if there are 4 semantic groups in total, and the similarities between the target data and the central data of each semantic group are 1, 2, 3 and 4 respectively, the semantic group corresponding to similarity 4 can be used as the target group.
[0147] In this way, the target data is obtained by querying the encoding data corresponding to the vocabulary to be evaluated, and the target group is selected based on the similarity between the target data and the central data of the semantic group. The data in the target group has a high similarity with the target data, thereby improving the accuracy of vocabulary association.
[0148] S14, classifying the coded data in the target group to obtain a plurality of candidate groups, each candidate group containing a plurality of coded data.
[0149] In an optional embodiment, classifying the data in the target group to obtain multiple candidate groups includes:
[0150] a. Classify the coded data in the target group according to a preset radius threshold and a preset density threshold to obtain a classification result, wherein the classification result includes core data and outlier data; in this optional embodiment, taking any coded data in the target group as an example, when the number of coded data contained in a range with the coded data as the center and the radius threshold as the radius is not less than the density threshold, the coded data is the core data, otherwise the coded data is the outlier data. In this solution, the radius threshold may be 0.1 and the density threshold may be 4. For example, if the number of coded data contained in a range with the coded data T as the center and the radius threshold 0.1 as the radius is not less than 4, the coded data T is the core data;
[0151] b. Set a discrimination condition based on the radius threshold, the discrimination condition means that the relationship between two coded data satisfies any one of the preset relationships, the preset relationships include: density direct, density reachable, and density connected. Density direct means that the Euclidean distance between the two coded data is not less than the radius threshold; density reachable means that the relationship between the two coded data is not density direct and the two coded data have common density direct coded data; density connected means that the relationship between the two coded data is not density reachable and they have common density reachable coded data. When the preset radius threshold is 0.5, the Euclidean distance between the encoded data E and the encoded data F is 0.4, then the relationship between the encoded data E and the encoded data F is density-reachable; if the Euclidean distance between the encoded data G and the encoded data F is 0.4, then the relationship between the encoded data G and the encoded data F is density-reachable, and when the Euclidean distance between the encoded data G and the encoded data E is greater than 0.5, then the relationship between the encoded data G and the encoded data E is density-reachable; when the relationship between the encoded data H and the encoded data E is not density-reachable, but the relationship between the encoded data H and the encoded data G is density-reachable, then the relationship between the encoded data H and the encoded data E is density-connected.
[0152] c, marking all core data as "not visited". In this optional embodiment, the coded data E, coded data F, coded data G, and coded data H may all be set as core data, and the coded data E, coded data F, coded data G, and coded data H may all be marked as "not visited".
[0153] d. Select any core data marked as “unvisited” as the target data;
[0154] e. traverse all the core data labeled as "unvisited" in sequence. If the traversed core data meets the discrimination condition, that is, meets any one of the preset relationships, the core data and the target data are classified into the same candidate group, and all the core data in the candidate group are marked as "visited"; in this optional embodiment, the coded data E is taken as the target data, and the traversed coded data H is taken as an example. Since the relationship between the coded data H and the target data E is density-connected, the coded data H meets the discrimination condition. The coded data H and the target data E are classified into the same candidate group, and all the coded data in the candidate group are marked as "visited";
[0155] f. Repeat steps d and e to obtain multiple candidate groups until all the encoded data in the target group are marked as "visited", then the classification is completed and multiple candidate groups are obtained, and all outlier data are taken as a candidate group.
[0156] In this way, by further classifying the coded data in the target group, multiple candidate groups are obtained, each candidate group contains multiple coded data, each coded data represents a word in the natural language database, and the coded data in each candidate group is used to represent words belonging to the same category, which can refine the results of vocabulary classification and thus improve the accuracy of vocabulary association.
[0157] S15, calculating a correlation index for each candidate group, where the correlation index is used to characterize the degree to which the coded data in each candidate group should be recommended.
[0158] In an optional embodiment, calculating the association index of each candidate group includes:
[0159] Calculate the mean of each candidate group, and use the encoded data with the smallest Euclidean distance to the mean in each candidate group as the centroid of each candidate group;
[0160] Calculate the variance of the encoded data in each candidate group as the degree of aggregation, where the degree of aggregation is used to represent the similarity of the words corresponding to the encoded data in the candidate group;
[0161] Calculating the similarity between the target data and the centroid data, wherein a higher similarity indicates that the target data is more similar to the centroid data;
[0162] The aggregation degree and the similarity are input into a custom normalization model to obtain a normalization result, and the normalization result is used as an association index for each candidate group, wherein the association index is used to characterize the degree of correlation between the encoded data in the candidate group and the target data.
[0163] In this optional embodiment, the mean of each candidate group may be calculated, and the coded data with the smallest Euclidean distance to the mean in each candidate group is used as the centroid of each candidate group. For example, when a candidate group includes 4 coded data and they are [0, 0.23, 0, 0], [0.45, 0, 0, 0], [0, 0, 0.65, 0] and [0, 0, 0, 0.55] respectively, the mean of the candidate group is calculated as follows:
[0164]
[0165] Then the mean is [0.11, 0.06, 0.16, 0.13]. Let the Euclidean distances between the four coded data and the mean be Dis1, Dis2, Dis3 and Dis4 respectively. Then the Euclidean distances between the four coded data and the mean are:
[0166]
[0167]
[0168]
[0169]
[0170] Then the encoding vector [0, 0.23, 0, 0] can be selected as the centroid of the candidate group.
[0171] In this optional embodiment, the variance of the coded data in each candidate group can be calculated as the aggregation degree of each candidate group. The higher the aggregation degree, the greater the influence of the words corresponding to the coded data in the candidate group on the semantics of the words in the target group, and the more important the words in the candidate group are and the more they should be recommended. The variance is calculated as follows:
[0172]
[0173] Among them, A represents a coded data in the candidate group; B represents the mean of the candidate group; j represents the index of the coded data in the candidate group; A j represents the jth coded data in the candidate group; k represents the number of coded data in the candidate group; x represents the variance of the candidate group, that is, the aggregation degree of the coded data in the candidate group. The higher the value of x, the lower the aggregation degree of the coded data in the candidate group, and the higher the influence of the vocabulary corresponding to the coded data in the candidate group on the semantics of this type of vocabulary.
[0174] For example, if a candidate group contains 4 coded data [0, 0.23, 0, 0], [0.45, 0, 0, 0], [0, 0, 0.65, 0] and [0, 0, 0, 0.55], and the mean of the coded data in the candidate group is [0.11, 0.06, 0.16, 0.13], then the variance of the coded data in the candidate group is calculated as follows:
[0175]
[0176] The degree of aggregation represented by the variance is 0.9534.
[0177] In this optional embodiment, the similarity between the target data and the centroid data may be calculated. The similarity may be the inverse of the Euclidean distance. The similarity may be denoted as y, and the calculation method thereof is:
[0178]
[0179] Wherein, y represents the similarity between the target data C and the centroid data D. The higher the value of y is, the higher the similarity between the vocabulary represented by the target data C and the vocabulary represented by the centroid data D is. i represents the dimension index of the target data and the centroid data, n represents the number of dimensions of the target data and the centroid data, and C i Represents the value of the i-th dimension in the target data, D i Represents the value of the i-th dimension in the centroid data.
[0180] Exemplarily, when the target data is [0, 0.23, 0, 0, 0] and the centroid data is [0, 0.45, 0, 0, 0], the similarity is calculated as follows:
[0181]
[0182] Then the similarity between the target data and the centroid data is 4.54.
[0183] In this optional embodiment, the customized normalization model satisfies the following relationship:
[0184]
[0185] Among them, T represents the association index. The higher the value of the association index is, the more the vocabulary corresponding to the coded data in the corresponding candidate group should be recommended; x represents the aggregation degree of the candidate group. The higher the aggregation degree is, the more important the vocabulary corresponding to the coded data in the candidate group is; y represents the similarity between the target data and the candidate group. The higher the similarity is, the more the vocabulary corresponding to the coded data in the candidate group should be recommended; α represents the preset harmonic constant. According to multiple test experiences, α can be 5; e represents the natural constant.
[0186] For example, when the harmonic constant is 5, the aggregation degree of a candidate group is 0.9534, and the similarity between the target data and the centroid data of the candidate group is 4.54, the calculation method of the association index of the candidate group is:
[0187]
[0188] Then the correlation index corresponding to all the coded data in the candidate group is 0.408.
[0189] In this way, by calculating the degree of aggregation of each candidate group, and respectively calculating the similarity between the centroid of each candidate group and the target data, and inputting the degree of aggregation and the similarity into a customized normalization model, the corresponding association index of each candidate group is obtained. The higher the value of the association index, the stronger the degree of association between the vocabulary represented by the encoded data in the corresponding candidate group and the vocabulary represented by the target data, thereby improving the accuracy of text recommendation.
[0190] S16, recommending words corresponding to the coded data in the candidate group in descending order of the association index.
[0191] In this optional embodiment, the association index of each candidate group can be compared with a preset threshold value to obtain a comparison result of each candidate group, and the comparison result includes greater than and not greater than. If the association index is greater than the preset threshold value, the coded data in the candidate group is retained, and if the association index is not greater than the preset threshold value, the coded data in the candidate group is deleted. Since the output result of the normalization model is a real number between 0 and 1, that is, the value of the association index is a real number between 0 and 1, the preset threshold value can be 0.5.
[0192] In this optional embodiment, the retained candidate groups can be sorted according to the association index. If the association index value of the candidate group is higher, the order of the candidate group will be higher. Further, the vocabulary corresponding to the encoded data in the candidate group can be recommended according to the order of the candidate group. The higher the order of the candidate group, the earlier the vocabulary corresponding to the encoded data in the candidate group will be recommended to the user.
[0193] In this way, the candidate groups are sorted according to their corresponding correlation indicators to obtain the order of the candidate groups, and the words corresponding to the encoded data in the candidate groups are recommended to the user according to the order, which can make a quantitative evaluation of the words and thus improve the accuracy of text recommendation.
[0194] The above-mentioned artificial intelligence-based text recommendation method obtains the quantitative expression of the text by encoding the vocabulary, which can improve the accuracy of text recommendation, and classifies the encoded data multiple times to obtain the optimized classification results. The target group data is further screened out from the optimized classification results based on the vocabulary to be evaluated, and then multiple candidate group data are obtained by secondary classification of the target group data, and the correlation indicators of each candidate group are evaluated, and vocabulary is recommended to users based on the correlation indicators, thereby improving the accuracy of text recommendation.
[0195] like Figure 2 , which is a functional module diagram of a preferred embodiment of an artificial intelligence-based text recommendation device provided in an embodiment of the present application. The artificial intelligence-based text recommendation device includes a word segmentation unit 110, an encoding unit 111, a first classification unit 112, a first calculation unit 113, a second classification unit 114, a second calculation unit 115, and a recommendation unit 116. The module / unit referred to in this application refers to a series of computer program segments that can be executed by a processor 13 and can perform fixed functions, which are stored in a memory 12. In this embodiment, the functions of each module / unit will be described in detail in subsequent embodiments.
[0196] In an optional embodiment, the word segmentation unit 110 is used to segment text in a natural language database using a preset word segmentation tool to obtain a plurality of words.
[0197] In this optional embodiment, the natural language database refers to a data set that is designed, stored, and managed according to a data structure for natural language texts. Exemplarily, the natural language database in this solution may be a MySQL database, which is an open source database whose function may be to store any natural language text.
[0198] In this optional embodiment, the preset word segmentation tool can be a jieba word segmentation tool, which is a code written by a Python program, and its function is to separate words in a natural language text to obtain multiple words. The form of the jieba word segmentation tool can be jieba.cut(sentence), where cut represents the word segmentation instruction in the jieba word segmentation tool, and sentence represents the natural language text.
[0199] Exemplarily, when the sentence is "everything looks like just waking up", the output of the jieba word segmentation tool is a list, each element in the list is a word, and the form of the list is [everything looks like just waking up].
[0200] In an optional embodiment, the encoding unit 111 is used to encode the words to obtain encoding data corresponding to each word.
[0201] In this optional embodiment, each word in the natural language database can be encoded according to a preset text encoding method. The preset text encoding method can be a one-hot algorithm, which means a one-hot encoding algorithm. The function of the one-hot encoding algorithm is to encode each word in the natural language database into binary vector data. Exemplarily, the binary vector data can be in the form of [0,1,0,0,0].
[0202] In this optional embodiment, the weight corresponding to each word in the natural language database can be calculated according to a preset word weight algorithm, and the weight is used to characterize the importance of each word in the natural language database. The higher the value of the weight, the more important the corresponding word is in the natural language database. The preset word weight algorithm can be a TF-IDF algorithm. The full name of the TF-IDF algorithm is term frequency-inverse document frequency, which means term frequency-inverse document frequency algorithm. The TF-IDF algorithm can output the weight corresponding to each word, and the value of the weight can be any real number.
[0203] In this optional embodiment, the product of the weight and the binary vector data of the corresponding vocabulary can be calculated to obtain the encoding data corresponding to each vocabulary. For example, if the binary data corresponding to the vocabulary "I" is [0, 1, 0, 0, 0] and its corresponding weight is 0.23, then the encoding data corresponding to the vocabulary "I" is [0, 0.23, 0, 0, 0].
[0204] In this optional embodiment, the vocabulary may be used as a key and the encoded data corresponding to the vocabulary may be used as a value to construct a key-value pair, and all the key-value pairs may be used as an encoded hash table.
[0205] Exemplarily, when the word is "I" and its corresponding encoding data is [0, 0.23, 0, 0, 0], the key-value pair is in the form of (I, [0, 0.23, 0, 0, 0]).
[0206] In an optional embodiment, the first classification unit 112 is configured to classify the coded data to obtain a plurality of semantic groups, each semantic group containing a plurality of coded data.
[0207] In this optional embodiment, the coded data is classified to obtain a plurality of semantic groups, each semantic group containing a plurality of coded data, including:
[0208] Constructing a center list according to a preset classification upper limit, and selecting a plurality of coded data as center data according to each element in the center list;
[0209] Classifying all the coded data multiple times according to the central data to obtain multiple classification results, each classification result comprising multiple vocabulary classification groups;
[0210] Calculate the sum of squared errors of each classification result as an evaluation value, and calculate the difference of the evaluation values to construct an evaluation result hash table;
[0211] The classification result corresponding to the minimum difference in the evaluation result hash table is searched, and all vocabulary classification groups in the classification result are used as semantic groups.
[0212] In this optional embodiment, the central list can be constructed according to a preset classification upper limit, where the preset classification upper limit is a positive integer greater than 2, and the classification upper limit is used to control the number of times the encoded data is classified.
[0213] In this optional embodiment, each element in the central list is a positive integer, wherein each element represents the number of categories obtained after classifying the encoded data each time. Since the encoded data needs to be divided into multiple categories, the minimum value of the elements in the central list is 2.
[0214] In this optional embodiment, each positive integer greater than or equal to 2 can be taken in turn as an element in the central list, and the preset upper limit of the classification can be taken as the last element in the central list. If the preset upper limit of the classification is n, the central list has n-1 elements.
[0215] Exemplarily, when the preset upper limit of classification is 9, the central list is [2, 3, 4, 5, 6, 7, 8, 9], where the value of each element is used to represent the number of categories obtained after each classification of the encoded data.
[0216] In this optional embodiment, the elements in the central list can be selected in sequence as the number of categories, and the encoded data equal to the number of categories can be randomly selected from the encoded data as the central data. For example, if the central list is [2,3,4,5,6,7,8,9], the first number of categories selected is 2, and two encoded data can be randomly selected from all the encoded data as the central data.
[0217] In an optional embodiment, classifying all the coded data multiple times according to the central data to obtain multiple classification results includes:
[0218] a, mark all coded data as 0 and randomly select a coded data as the current data;
[0219] b, respectively calculating the Euclidean distance between the current data and each center data;
[0220] c. Mark the current data as 1 and classify it into the category where the central data corresponding to the smaller Euclidean distance belongs;
[0221] d. Randomly select a coded data marked as 0 as the current data again, and repeat steps b to d until all coded data are marked as 1, then stop the iteration and obtain multiple vocabulary classification groups;
[0222] e. Calculate the mean of all the coded data in each vocabulary classification group. If all the means meet the preset judgment conditions, the vocabulary classification group is taken as the classification result. Otherwise, the coded data closest to the mean in each vocabulary classification group is taken as the central data and steps a to e are repeated to obtain the classification result.
[0223] In this optional embodiment, the specific implementation steps of performing multiple classifications on all the encoded data to obtain multiple classification results are:
[0224] A1: Mark all coded data as 0, and randomly select a coded data as current data, where the current data corresponds to a certain word in the natural language database;
[0225] A2: Calculate the Euclidean distance between the current data and each center data respectively. For example, if the current data is [0.23, 0, 0, 0] and a center data is [0, 0.65, 0, 0], the Euclidean distance between the current data and the center data is calculated as follows:
[0226]
[0227] Then the Euclidean distance between the current data and the central data is 0.4754.
[0228] The Euclidean distance is used to represent the similarity between the current data and the central data. The smaller the Euclidean distance is, the higher the similarity between the vocabulary represented by the current data and the vocabulary represented by the central data is.
[0229] A3: Mark the current data as 1 and classify it into the vocabulary classification group where the central data corresponding to the smaller Euclidean distance belongs. For example, if there are four central data in total, and the Euclidean distance between the current data and each central data is recorded as Dis 1 =0.4754, Dis 2 =0.57, Dis 3 =0.65 and Dis 4 =0.32, where the superscript of each Euclidean distance represents the index of each center data. Since the minimum value of the four Euclidean distances is Dis 4 =0.32, so the current data is classified into the vocabulary classification group where the fourth central data is located;
[0230] A4: randomly select a coded data marked as 0 as the current data, and repeat steps A2 to A4 until all coded data are marked as 1 and multiple vocabulary classification groups are obtained;
[0231] A5: Calculate the mean of all the coded data in each vocabulary classification group. If all the means meet the preset discrimination conditions, the multiple vocabulary classification groups are taken as the classification results. Otherwise, the coded data closest to the mean in each vocabulary classification group is taken as the central data, and repeat steps A1 to A5 to obtain the classification result.
[0232] The preset judgment condition refers to that the Euclidean distance between the mean and the central data is less than a preset threshold. For example, the preset threshold may be 0.1. The purpose of this step is to ensure that the classification result changes are small enough, that is, the performance of the classification result is relatively excellent.
[0233] In this optional embodiment, each element in the central list corresponds to a classification result, each classification result corresponds to multiple vocabulary classification groups, each vocabulary classification group contains multiple encoding data, each encoding data corresponds to a vocabulary, and the encoding data in each vocabulary classification group corresponds to a class of vocabulary with the same semantics.
[0234] In an optional embodiment, the sum of squared errors of each classification result is calculated as an evaluation value, and the difference of the evaluation values is calculated to construct an evaluation result hash table, including:
[0235] Calculate the error of each vocabulary classification group respectively, wherein the error is the sum of the squares of the Euclidean distances between all the encoded data and the central data in each vocabulary classification group;
[0236] Calculate the sum of all errors in each classification result as the evaluation value of each classification result;
[0237] The evaluation values are arranged from small to large according to the number of vocabulary classification groups in each classification result, and after calculating the difference between two adjacent evaluation values, the differences are combined according to the order of the evaluation values to construct an evaluation result hash table.
[0238] In this optional embodiment, the error of each vocabulary classification group in each classification result can be calculated respectively, and the error is the sum of the squares of the Euclidean distances between all the encoded data and the central data in each vocabulary classification group. The Euclidean distance is calculated as follows:
[0239]
[0240] Where A represents the central data, B represents the coded data, and i represents the index of the dimension of the central data and the coded data. Therefore, A i represents the value of the i-th dimension of the central data, B i represents the value of the i-th dimension of the encoded data, m represents the total number of dimensions of the central data and the encoded data, Dis represents the Euclidean distance, and the smaller the Euclidean distance is, the higher the similarity between the central data and the encoded data is, and the more similar the vocabulary represented by the central data is to the vocabulary represented by the encoded data.
[0241] For example, if the central data of a vocabulary classification group is [0, 0.23, 0, 0] and one of the coded data is [0.45, 0, 0, 0], the Euclidean distance between the central data and the coded data is calculated as follows:
[0242]
[0243] Then the Euclidean distance between the central data and the encoded data is 0.5.
[0244] For example, if a vocabulary classification group includes four coded data and one central data, and the Euclidean distances between the four coded data and the central data are 0.5, 1, 1.5 and 2 respectively, then the error corresponding to the vocabulary classification group is calculated as follows:
[0245] 0.5 2 +1 2 +1.5 2 +2 2 =7.5
[0246] The error corresponding to the vocabulary classification group is 7.5. The smaller the error is, the higher the degree of aggregation of all the coded data in the vocabulary classification group is, and the more similar the vocabularies corresponding to all the coded data in the vocabulary classification group are.
[0247] In this optional embodiment, the sum of all errors in each classification result can be calculated as the evaluation value of each classification result. The smaller the evaluation value, the better the effect of the corresponding classification result. For example, if a classification result includes four vocabulary classification groups, and the errors of the four vocabulary classification groups are 7.5, 9.5, 10.5 and 11.5 respectively, the evaluation value corresponding to the classification result is 39.
[0248] In this optional embodiment, the number of vocabulary classification groups in each classification result can be used as a key and the evaluation value corresponding to each classification result can be used as a value to construct a key-value pair, and the key-value pairs are arranged from small to large according to the value of the key.
[0249] For example, if the corresponding evaluation value is 100 when the number of vocabulary classification groups is 2, the corresponding evaluation value is 60 when the number of vocabulary classification groups is 3, and the corresponding evaluation value is 50 when the number of vocabulary classification groups is 4, then the arrangement order of the key-value pairs is (2, 100), (3, 60), and (4, 50).
[0250] In this optional embodiment, the difference between two adjacent evaluation values may be calculated. For example, if the evaluation value obtained when the number of vocabulary classification groups is 2 is 100, and the evaluation value obtained when the number of vocabulary classification groups is 3 is 60, then the difference is 40.
[0251] In this optional embodiment, the differences can be combined according to the order of the evaluation values to construct an evaluation result hash table. For example, if the evaluation value differences are 40 and 10 respectively, the evaluation result hash table is [([2,3],40),([3,4],10)].
[0252] In this optional embodiment, the key corresponding to the smallest value in the evaluation result hash table can be selected as the target key. For example, if the evaluation result hash table is [([2,3],40),([3,4],10)], the smallest value is 10 and the corresponding key is [3,4].
[0253] In this optional embodiment, the vocabulary classification group in the classification result corresponding to the smaller value in the target key can be selected as the semantic group.
[0254] For example, if the target key is [3, 4], three vocabulary classification groups in the classification result corresponding to 3 can be selected as semantic groups.
[0255] In an optional embodiment, the first calculation unit 113 calculates the similarity between the encoded data corresponding to the vocabulary to be evaluated and each semantic group, and takes the semantic group corresponding to the maximum similarity as the target group.
[0256] In this optional embodiment, the vocabulary to be evaluated refers to the vocabulary that needs to be associated in the natural language database. The vocabulary to be evaluated can be used as a key, and the encoding hash table can be traversed in sequence to find the value corresponding to the key as the target data.
[0257] Exemplarily, when the word to be evaluated is "description", the word "description" can be used as a key, and all keys in the encoding hash table can be traversed in turn to query the encoded data corresponding to the word "description", and the encoded data corresponding to the word "description" can be used as target data. The target data can be in the form of [0, 0.23, 0, 0, 0].
[0258] In this optional embodiment, the similarity between the target data and the central data of each semantic group may be used as the similarity between the target data and the semantic group, and the semantic group corresponding to the minimum similarity is selected as the target group.
[0259] In this optional embodiment, the Euclidean distance between the target data and the central data of each semantic group can be calculated, and the inverse of the Euclidean distance can be used as the similarity. The higher the similarity, the more similar the target data is to the central data of the semantic group, and the higher the similarity between the vocabulary represented by the target data and the vocabulary in the semantic group.
[0260] In this optional embodiment, if the target data is A and the semantic group center data is B, the similarity is calculated as follows:
[0261]
[0262] Wherein, S represents the similarity between the target data A and the semantic group central data B. The higher the value of S, the higher the similarity between the vocabulary represented by the target data A and the vocabulary represented by the central data B. i represents the dimension index of the target data and the central data, n represents the number of dimensions of the target data and the central data, and A i represents the value of the i-th dimension in the target data, B i Represents the value of the i-th dimension in the semantic group center data.
[0263] Exemplarily, when the target data is [0, 0.23, 0, 0, 0] and the semantic group center data is [0, 0.45, 0, 0, 0], the similarity is calculated as follows:
[0264]
[0265] In this optional embodiment, the semantic group corresponding to the greatest similarity can be used as the target group. For example, if there are 4 semantic groups in total, and the similarities between the target data and the central data of each semantic group are 1, 2, 3 and 4 respectively, the semantic group corresponding to similarity 4 can be used as the target group.
[0266] In an optional embodiment, the second classification unit 114 is used to classify the coded data in the target group to obtain multiple candidate groups, each candidate group containing multiple coded data.
[0267] In an optional embodiment, classifying the data in the target group to obtain multiple candidate groups includes:
[0268] a. Classify the coded data in the target group according to a preset radius threshold and a preset density threshold to obtain a classification result, wherein the classification result includes core data and outlier data; in this optional embodiment, taking any coded data in the target group as an example, when the number of coded data contained in a range with the coded data as the center and the radius threshold as the radius is not less than the density threshold, the coded data is the core data, otherwise the coded data is the outlier data. In this solution, the radius threshold may be 0.1 and the density threshold may be 4. For example, if the number of coded data contained in a range with the coded data T as the center and the radius threshold 0.1 as the radius is not less than 4, the coded data T is the core data;
[0269] b. Set a discrimination condition based on the radius threshold, the discrimination condition means that the relationship between two coded data satisfies any one of the preset relationships, the preset relationships include: density direct, density reachable, and density connected. Density direct means that the Euclidean distance between the two coded data is not less than the radius threshold; density reachable means that the relationship between the two coded data is not density direct and the two coded data have common density direct coded data; density connected means that the relationship between the two coded data is not density reachable and they have common density reachable coded data. When the preset radius threshold is 0.5, the Euclidean distance between the encoded data E and the encoded data F is 0.4, then the relationship between the encoded data E and the encoded data F is density-reachable; if the Euclidean distance between the encoded data G and the encoded data F is 0.4, then the relationship between the encoded data G and the encoded data F is density-reachable, and when the Euclidean distance between the encoded data G and the encoded data E is greater than 0.5, then the relationship between the encoded data G and the encoded data E is density-reachable; when the relationship between the encoded data H and the encoded data E is not density-reachable, but the relationship between the encoded data H and the encoded data G is density-reachable, then the relationship between the encoded data H and the encoded data E is density-connected.
[0270] c, marking all core data as "not visited". In this optional embodiment, the coded data E, coded data F, coded data G, and coded data H may all be set as core data, and the coded data E, coded data F, coded data G, and coded data H may all be marked as "not visited".
[0271] d. Select any core data marked as “unvisited” as the target data;
[0272] e. traverse all the core data labeled as "unvisited" in sequence. If the traversed core data meets the discrimination condition, that is, meets any one of the preset relationships, the core data and the target data are classified into the same candidate group, and all the core data in the candidate group are marked as "visited"; in this optional embodiment, the coded data E is taken as the target data, and the traversed coded data H is taken as an example. Since the relationship between the coded data H and the target data E is density-connected, the coded data H meets the discrimination condition. The coded data H and the target data E are classified into the same candidate group, and all the coded data in the candidate group are marked as "visited";
[0273] f. Repeat steps d and e to obtain multiple candidate groups until all the encoded data in the target group are marked as "visited", then the classification is completed and multiple candidate groups are obtained, and all outlier data are taken as a candidate group.
[0274] In an optional embodiment, the second calculation unit 115 is used to calculate a relevance index for each candidate group, where the relevance index is used to characterize the degree to which the coded data in each candidate group should be recommended.
[0275] In an optional embodiment, calculating the association index of each candidate group includes:
[0276] Calculate the mean of each candidate group, and use the encoded data with the smallest Euclidean distance to the mean in each candidate group as the centroid of each candidate group;
[0277] Calculate the variance of the encoded data in each candidate group as the degree of aggregation, where the degree of aggregation is used to represent the similarity of the words corresponding to the encoded data in the candidate group;
[0278] Calculating the similarity between the target data and the centroid data, wherein a higher similarity indicates that the target data is more similar to the centroid data;
[0279] The aggregation degree and the similarity are input into a custom normalization model to obtain a normalization result, and the normalization result is used as an association index for each candidate group, wherein the association index is used to characterize the degree of correlation between the encoded data in the candidate group and the target data.
[0280] In this optional embodiment, the mean of each candidate group may be calculated, and the coded data with the smallest Euclidean distance to the mean in each candidate group is used as the centroid of each candidate group. For example, when a candidate group includes 4 coded data and they are [0, 0.23, 0, 0], [0.45, 0, 0, 0], [0, 0, 0.65, 0] and [0, 0, 0, 0.55] respectively, the mean of the candidate group is calculated as follows:
[0281]
[0282] Then the mean is [0.11, 0.06, 0.16, 0.13]. Let the Euclidean distances between the four coded data and the mean be Dis1, Dis2, Dis3 and Dis4 respectively. Then the Euclidean distances between the four coded data and the mean are:
[0283]
[0284]
[0285]
[0286]
[0287] Then the encoding vector [0, 0.23, 0, 0] can be selected as the centroid of the candidate group.
[0288] In this optional embodiment, the variance of the coded data in each candidate group can be calculated as the aggregation degree of each candidate group. The higher the aggregation degree, the greater the influence of the words corresponding to the coded data in the candidate group on the semantics of the words in the target group, and the more important the words in the candidate group are and the more they should be recommended. The variance is calculated as follows:
[0289]
[0290] Among them, A represents a coded data in the candidate group; B represents the mean of the candidate group; j represents the index of the coded data in the candidate group; A j represents the jth coded data in the candidate group; k represents the number of coded data in the candidate group; x represents the variance of the candidate group, that is, the aggregation degree of the coded data in the candidate group. The higher the value of x, the lower the aggregation degree of the coded data in the candidate group, and the higher the influence of the vocabulary corresponding to the coded data in the candidate group on the semantics of this type of vocabulary.
[0291] For example, if a candidate group contains 4 coded data [0, 0.23, 0, 0], [0.45, 0, 0, 0], [0, 0, 0.65, 0] and [0, 0, 0, 0.55], and the mean of the coded data in the candidate group is [0.11, 0.06, 0.16, 0.13], then the variance of the coded data in the candidate group is calculated as follows:
[0292]
[0293] The degree of aggregation represented by the variance is 0.9534.
[0294] In this optional embodiment, the similarity between the target data and the centroid data may be calculated. The similarity may be the inverse of the Euclidean distance. The similarity may be denoted as y, and the calculation method thereof is:
[0295]
[0296] Wherein, y represents the similarity between the target data C and the centroid data D. The higher the value of y is, the higher the similarity between the vocabulary represented by the target data C and the vocabulary represented by the centroid data D is. i represents the dimension index of the target data and the centroid data, n represents the number of dimensions of the target data and the centroid data, and C i Represents the value of the i-th dimension in the target data, D i Represents the value of the i-th dimension in the centroid data.
[0297] Exemplarily, when the target data is [0, 0.23, 0, 0, 0] and the centroid data is [0, 0.45, 0, 0, 0], the similarity is calculated as follows:
[0298]
[0299] Then the similarity between the target data and the centroid data is 4.54.
[0300] In this optional embodiment, the customized normalization model satisfies the following relationship:
[0301]
[0302] Among them, T represents the association index. The higher the value of the association index is, the more the vocabulary corresponding to the coded data in the corresponding candidate group should be recommended; x represents the aggregation degree of the candidate group. The higher the aggregation degree is, the more important the vocabulary corresponding to the coded data in the candidate group is; y represents the similarity between the target data and the candidate group. The higher the similarity is, the more the vocabulary corresponding to the coded data in the candidate group should be recommended; α represents the preset harmonic constant. According to multiple test experiences, α can be 5; e represents the natural constant.
[0303] Exemplarily, when the harmonic constant is 5, the aggregation degree of a candidate group is 0.9534, and the similarity between the target data and the centroid data of the candidate group is 4.54, the calculation method of the association index of the candidate group is:
[0304]
[0305] Then the correlation index corresponding to all the coded data in the candidate group is 0.408.
[0306] In an optional embodiment, the recommendation unit 116 is configured to recommend the vocabulary corresponding to the encoding data in the candidate group in descending order of the association index.
[0307] In this optional embodiment, the association index of each candidate group can be compared with a preset threshold value to obtain a comparison result of each candidate group, and the comparison result includes greater than and not greater than. If the association index is greater than the preset threshold value, the coded data in the candidate group is retained, and if the association index is not greater than the preset threshold value, the coded data in the candidate group is deleted. Since the output result of the normalization model is a real number between 0 and 1, that is, the value of the association index is a real number between 0 and 1, the preset threshold value can be 0.5.
[0308] In this optional embodiment, the retained candidate groups can be sorted according to the association index. If the association index value of the candidate group is higher, the order of the candidate group will be higher. Further, the vocabulary corresponding to the encoded data in the candidate group can be recommended according to the order of the candidate group. The higher the order of the candidate group, the earlier the vocabulary corresponding to the encoded data in the candidate group will be recommended to the user.
[0309] The above-mentioned artificial intelligence-based text recommendation method obtains the quantitative expression of the text by encoding the vocabulary, which can improve the accuracy of text recommendation, and classifies the encoded data multiple times to obtain the optimized classification results. The target group data is further screened out from the optimized classification results based on the vocabulary to be evaluated, and then multiple candidate group data are obtained by secondary classification of the target group data, and the correlation indicators of each candidate group are evaluated, and vocabulary is recommended to users based on the correlation indicators, thereby improving the accuracy of text recommendation.
[0310] like Figure 3 , which is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 1 includes a memory 12 and a processor 13. The memory 12 is used to store computer-readable instructions, and the processor 13 executes the computer-readable instructions stored in the memory to implement the text recommendation method based on artificial intelligence in any of the above embodiments.
[0311] In an optional embodiment, the electronic device 1 further includes a bus, a computer program stored in the memory 12 and executable on the processor 13, such as an artificial intelligence-based text recommendation program.
[0312] Figure 3 Only the electronic device 1 having components 12-13 is shown, and those skilled in the art will understand that Figure 3 The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0313] Combination Figure 1 The memory 12 in the electronic device 1 stores a plurality of computer-readable instructions to implement a text recommendation method based on artificial intelligence, and the processor 13 can execute the plurality of instructions to implement:
[0314] Use a preset word segmentation tool to segment the text in the natural language database to obtain multiple words;
[0315] Encode the words in the natural language database to obtain encoding data corresponding to each word;
[0316] Classifying the coded data to obtain a plurality of semantic groups, each semantic group comprising a plurality of coded data;
[0317] Calculate the similarity between the encoded data corresponding to the vocabulary to be evaluated and each semantic group, and take the semantic group corresponding to the maximum similarity as the target group;
[0318] Classifying the coded data in the target group to obtain a plurality of candidate groups, each candidate group containing a plurality of coded data;
[0319] Calculating a correlation index for each candidate group, where the correlation index is used to characterize the degree to which the encoded data in each candidate group should be recommended;
[0320] The words corresponding to the coded data in the candidate group are recommended in descending order of the association index.
[0321] Specifically, the specific implementation method of the processor 13 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.
[0322] Those skilled in the art will appreciate that the schematic diagram is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. The electronic device 1 may have either a bus structure or a star structure. The electronic device 1 may also include more or less other hardware or software than shown in the diagram, or a different arrangement of components. For example, the electronic device 1 may also include input and output devices, network access devices, etc.
[0323] It should be noted that the electronic device 1 is only an example, and other existing or future electronic products that are suitable for the present application should also be included in the protection scope of the present application and included here by reference.
[0324] Among them, the memory 12 includes at least one type of readable storage medium, and the readable storage medium can be non-volatile or volatile. The readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 12 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 12 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device 1. Further, the memory 12 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 12 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of a text recommendation program based on artificial intelligence, but also can be used to temporarily store data that has been output or is to be output.
[0325] In some embodiments, the processor 13 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 13 is the control core (Control Unit) of the electronic device 1, and uses various interfaces and lines to connect the various components of the entire electronic device 1, and executes or executes programs or modules stored in the memory 12 (for example, executing a text recommendation program based on artificial intelligence, etc.), and calls data stored in the memory 12 to execute various functions of the electronic device 1 and process data.
[0326] The processor 13 executes the operating system of the electronic device 1 and various installed applications. The processor 13 executes the applications to implement the steps in the above-mentioned embodiments of the text recommendation method based on artificial intelligence, for example Figure 1 Steps shown.
[0327] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 13 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into a word segment unit 110, an encoding unit 111, a first classification unit 112, a first calculation unit 113, a second classification unit 114, a second calculation unit 115, and a recommendation unit 116.
[0328] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a computer device, or a network device, etc.) or a processor to execute a part of the text recommendation method based on artificial intelligence described in each embodiment of the present application.
[0329] If the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also instruct the relevant hardware devices to complete through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented.
[0330] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory and other memory, etc.
[0331] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0332] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. Figure 3 Only one arrow is used in the figure, but it does not mean that there is only one bus or one type of bus. The bus is configured to realize the connection and communication between the memory 12 and at least one processor 13, etc.
[0333] An embodiment of the present application also provides a computer-readable storage medium (not shown), in which computer-readable storage medium is stored computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the artificial intelligence-based text recommendation method described in any of the above embodiments.
[0334] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0335] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the specification can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.
[0336] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.
Claims
1. A text recommendation method based on artificial intelligence, characterized in that: The method comprises: Use a preset word segmentation tool to segment the text in the natural language database to obtain multiple words; Encoding the words to obtain encoding data corresponding to each word; Classify the coded data to obtain multiple semantic groups, each semantic group contains multiple coded data; wherein, the classifying the coded data to obtain multiple semantic groups includes: constructing a central list according to a preset classification upper limit, each element in the central list is a positive integer, and selecting multiple coded data as central data according to each element in the central list; classifying all coded data multiple times according to the central data to obtain multiple classification results, each classification result contains multiple vocabulary classification groups; calculating the sum of square errors of each classification result as an evaluation value, and calculating the difference of the evaluation values to construct an evaluation result hash table; searching for the classification result corresponding to the minimum difference in the evaluation result hash table, and taking all vocabulary classification groups in the classification result as semantic groups; wherein, the classifying of all coded data multiple times according to the central data Classification to obtain multiple classification results, each classification result includes multiple vocabulary classification groups, including: a, marking all coded data as 0 and randomly selecting a coded data as current data; b, respectively calculating the Euclidean distance between the current data and each central data; c, marking the current data as 1 and classifying it into the category where the central data corresponding to the smaller Euclidean distance is located; d, randomly selecting a coded data marked as 0 as current data again, and repeating steps b to d until all coded data are marked as 1, then stopping iteration and obtaining multiple vocabulary classification groups; e, calculating the mean of all coded data in each vocabulary classification group, if the mean meets a preset discrimination condition, taking the vocabulary classification group as the classification result, otherwise taking the coded data closest to the mean in each vocabulary classification group as the central data and repeating steps a to e to obtain the classification result; Calculate the similarity between the encoded data corresponding to the vocabulary to be evaluated and each semantic group, and take the semantic group corresponding to the maximum similarity as the target group; Classifying the coded data in the target group to obtain a plurality of candidate groups, each candidate group containing a plurality of coded data; Calculating a correlation index for each candidate group, where the correlation index is used to characterize the degree to which the encoded data in each candidate group should be recommended; The words corresponding to the coded data in the candidate group are recommended in descending order of the association index.
2. The text recommendation method based on artificial intelligence as claimed in claim 1, characterized in that: The step of calculating the sum of square errors of each classification result as an evaluation value, and calculating the difference of the evaluation values to construct an evaluation result hash table includes: Calculate the error of each vocabulary classification group respectively, wherein the error is the sum of the squares of the Euclidean distances between all the encoded data and the central data in each vocabulary classification group; Calculate the sum of errors of all vocabulary classification groups in each classification result as the evaluation value of each classification result; The evaluation values are arranged from small to large according to the number of vocabulary classification groups in each classification result, and after calculating the difference between two adjacent evaluation values, the differences are combined according to the order of the evaluation values to construct an evaluation result hash table.
3. The text recommendation method based on artificial intelligence as claimed in claim 1, characterized in that: The classifying the coded data in the target group to obtain a plurality of candidate groups comprises: a, classifying the coded data in the target group according to a preset radius threshold and a preset density threshold to obtain a classification result, wherein the classification result includes core data and outlier data; b. Setting a discrimination condition according to the radius threshold, wherein the discrimination condition refers to that the relationship between two data satisfies any one of the preset relationships, wherein the preset relationships include: density direct, density reachable, and density connected, wherein the density direct means that the Euclidean distance between two coded data is not less than the radius threshold; the density reachable means that the relationship between two coded data is not density direct and the two coded data have common density direct coded data; the density connected means that the relationship between two coded data is not density reachable and the two coded data have common density reachable coded data; c, mark all core data as "not accessed"; d. Select any core data marked as "unvisited" as the target data; e. traverse all core data labeled as "unvisited" in sequence. If the traversed core data meets the discrimination condition, the core data and the target data are classified into the same candidate group, and all core data in the candidate group are marked as "visited"; f. Repeat steps d and e to obtain multiple candidate groups, and take all outlier data as a candidate group.
4. The text recommendation method based on artificial intelligence as claimed in claim 1, characterized in that: Calculating the association index of each candidate group includes: Calculate the mean of each candidate group, and take the encoded data with the smallest Euclidean distance from the mean in each candidate group as the centroid of each candidate group; Calculate the variance of the encoded data in each candidate group as the degree of aggregation, where the degree of aggregation is used to represent the similarity of the words corresponding to the encoded data in the candidate group; Calculating the similarity between the target data and the centroid data, wherein the higher the similarity, the more similar the target data is to the centroid data; The aggregation degree and the similarity are input into a custom normalization model to obtain a normalization result, and the normalization result is used as an association index for each candidate group, wherein the association index is used to characterize the degree of correlation between the encoded data in the candidate group and the target data.
5. The text recommendation method based on artificial intelligence as claimed in claim 4, characterized in that: The custom normalization model satisfies the relationship: Wherein, T represents the association index, and the higher the value of the association index is, the more the vocabulary corresponding to the coded data in the corresponding candidate group should be recommended; x represents the aggregation degree of the candidate group, and the higher the aggregation degree is, the more important the vocabulary corresponding to the coded data in the candidate group is; y represents the similarity between the target data and the candidate group, and the higher the similarity is, the more the vocabulary corresponding to the coded data in the candidate group should be recommended; represents the preset harmonic constant; e represents the natural constant.
6. A text recommendation device based on artificial intelligence, the device comprising a unit for implementing the method according to any one of claims 1 to 5, characterized in that: The device comprises: A word segmentation unit, used to segment text in a natural language database using a preset word segmentation tool to obtain multiple words; An encoding unit, used for encoding the words in the natural language database to obtain encoding data corresponding to each word; A first classification unit, configured to classify the coded data to obtain a plurality of semantic groups, each semantic group comprising a plurality of coded data; A first calculation unit is used to calculate the similarity between the encoded data corresponding to the vocabulary to be evaluated and each semantic group, and take the semantic group corresponding to the maximum similarity as the target group; A second classification unit, used for classifying the coded data in the target group to obtain a plurality of candidate groups, each candidate group containing a plurality of coded data; A second calculation unit, used to calculate a relevance index for each candidate group, wherein the relevance index is used to characterize the degree to which the coded data in each candidate group should be recommended; The recommendation unit is used to recommend the words corresponding to the coded data in the candidate group in descending order of the association index.
7. An electronic device, characterized in that: The electronic device comprises: a memory storing computer readable instructions; and A processor executes computer-readable instructions stored in the memory to implement the artificial intelligence-based text recommendation method as described in any one of claims 1 to 5.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions are executed by a processor in an electronic device to implement the artificial intelligence-based text recommendation method as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Text recommendation method and system, terminal and storage medium
CN114239572A
Text clustering method and apparatus
WO2021143009A1