Information processing method, electronic device, and computer program product

Abstract

Embodiments of the present disclosure provide an information processing method, an electronic device, and a computer program product. In the information processing method, a computing device determines a target object associated with a text among a predetermined plurality of objects based on a plurality of weights corresponding to a plurality of terms in the text, and further determines a set of key terms among the plurality of terms with respect to the determined target object. Then, the computing device determines a set of target terms related to a text topic of the text among the set of key terms. The computing device then outputs the set of target terms in association with an identifier of the target object. In this way, the credibility of the target object associated with the text determined by the information processing method is improved, thereby improving the user experience of the information processing method.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to information technology, computer technology or storage technology, and more particularly to an information processing method, electronic device and computer program product. Background Technology

[0002] In a broad sense, a recommender system is an information filtering or processing system that recommends one or more objects to a user based on the information the user inputs. For example, the objects recommended by a recommender system can include multimedia content, news, books, academic papers, knowledge bases, data or information databases, search queries, category-based classifications, and other products or information.

[0003] Recommender systems can be used in many different scenarios. For example, in scenarios related to storage systems, a recommender system can be used in the storage system's customer service (or technical support) system. Specifically, users of the storage system may encounter problems while using the system and provide the customer service system with text describing the problem. This text can be input into the recommender system, which can then recommend knowledge or information bases to the storage system's technical support personnel to help them solve the problem. Summary of the Invention

[0004] The embodiments of this disclosure propose a technical solution for determining a target object and a set of target words associated with input text, which can improve the credibility of the target object. More specifically, the embodiments of this disclosure provide an information processing method, an electronic device, and a computer program product.

[0005] In a first aspect of this disclosure, an information processing method is provided. The method includes: determining a target object associated with the text from among predetermined multiple objects based on multiple weights corresponding to multiple words in a text, and a set of keywords related to the target object from among the multiple words. The method further includes: determining a set of target words related to the text's topic from the set of keywords. The method further includes: outputting the set of target words in association with an identifier of the target object.

[0006] In a second aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processor and at least one memory storing computer program instructions. The at least one memory and the computer program instructions are configured, together with the at least one processor, to cause the electronic device to: determine, among predetermined multiple objects, a target object associated with the text, and a set of keywords related to determining the target object from among the multiple words, based on multiple weights corresponding to multiple words in the text. The at least one memory and the computer program instructions are also configured, together with the at least one processor, to cause the electronic device to: determine, within the set of keywords, a set of target words related to the textual topic of the text. The at least one memory and the computer program instructions are further configured, together with the at least one processor, to cause the electronic device to: output the set of target words in association with an identifier of the target object.

[0007] In a third aspect of this disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-volatile computer-readable medium and includes machine-executable instructions. When executed, the machine-executable instructions cause a machine to perform the steps of the method according to the first aspect.

[0008] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other objects, features, and advantages of embodiments of the present disclosure will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of the present disclosure are illustrated in the drawings by way of example and not limitation.

[0010] Figure 1 A schematic diagram of an example information processing environment in which embodiments of the present disclosure may be implemented is shown.

[0011] Figure 2 A schematic diagram is shown illustrating how, in an embodiment of this disclosure, weights are assigned to multiple vectors to compute a result vector.

[0012] Figure 3 A schematic diagram illustrating attention weight visualization according to an embodiment of the present disclosure is shown.

[0013] Figure 4 A flowchart of an example information processing method according to an embodiment of the present disclosure is shown.

[0014] Figure 5 A flowchart illustrating an example process for determining a set of keywords according to embodiments of the present disclosure is shown.

[0015] Figure 6 A schematic diagram is shown illustrating the determination of a weight average based on a plurality of weight components according to an embodiment of the present disclosure.

[0016] Figure 7 A schematic diagram is shown illustrating the determination of a set of keywords based on the average weight of words according to an embodiment of the present disclosure.

[0017] Figure 8 A flowchart illustrating an example process for determining a target word set according to an embodiment of the present disclosure is shown.

[0018] Figure 9 A schematic diagram is shown illustrating a method for determining a target word set based on multiple text topic relevances between multiple words and text topics, according to an embodiment of the present disclosure.

[0019] Figure 10 A schematic diagram illustrating the use of a topic model to determine the topic distribution of text according to an embodiment of the present disclosure is shown.

[0020] Figure 11 A flowchart illustrating an example process for determining the text topic relevance between words in a keyword set and a text topic, according to embodiments of the present disclosure, is shown.

[0021] Figure 12 A schematic diagram is shown illustrating a text topic representation as a topic distribution about a plurality of predetermined topics according to an embodiment of the present disclosure.

[0022] Figure 13 A schematic diagram illustrating a method for determining text topic relevance based on multiple predetermined topic relevances between words and multiple predetermined topics, according to an embodiment of the present disclosure.

[0023] Figure 14 A flowchart illustrating an example process for determining the relevance of words in a keyword set to a predetermined topic according to embodiments of the present disclosure is shown.

[0024] Figure 15 A schematic diagram is shown illustrating a predetermined topic including a set of words and a set of probabilities, and a set of word vectors corresponding to the set of words, according to an embodiment of the present disclosure.

[0025] Figure 16 A schematic diagram illustrating the determination of predetermined topic relevance based on predetermined topic vectors and word vectors according to an embodiment of the present disclosure is shown.

[0026] Figure 17 A schematic framework diagram of a recommender system that simultaneously outputs recommended objects and explanatory information according to embodiments of the present disclosure is shown.

[0027] Figure 18A comparison is shown between the target word set output by the information processing method according to an embodiment of the present disclosure and the keywords output by other keyword determination schemes.

[0028] Figure 19 A block diagram of an example device that can be used to implement embodiments of the present disclosure is shown.

[0029] Throughout all the accompanying drawings, the same or similar reference numerals are used to denote the same or similar components. Detailed Implementation

[0030] The principles and spirit of this disclosure will now be described with reference to several exemplary embodiments illustrated in the accompanying drawings. It should be understood that these embodiments are described merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. In the description and claims herein, unless otherwise defined, all technical and scientific terms used herein have the meanings commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0031] As mentioned above, recommender systems can be used in many different scenarios. For example, in scenarios related to storage systems, recommender systems can be used in the storage system's customer service (or technical support) system. Specifically, users of the storage system may encounter problems while using the system and provide the customer service system with text describing the problem. This text can be input into the recommender system, which can then recommend knowledge or information bases to the storage system's technical support personnel to help them solve the problem.

[0032] However, conventional recommender systems only provide users with recommendations, without offering effective solutions to increase user trust in these recommendations. This can lead to users distrusting the recommended results or objects, resulting in a poor user experience and reduced efficiency. More specifically, when recommending an object, recommender systems typically don't provide explanations for why it was recommended. This lack of explanation can lower user trust in the recommendations, causing them to question their accuracy or validity.

[0033] For example, in the scenario described above where a recommendation system is used in a customer service system for a storage system, if the recommendation system only recommends knowledge or information bases for solving storage system problems to technical support personnel without explaining the reasons for recommending such knowledge or information bases, the technical support personnel may question whether the recommended knowledge or information bases can really solve the storage system problems. As a result, they may give up further access to the knowledge or information bases recommended by the recommendation system, which leads to a decrease in the user experience of the technical support personnel using the recommendation system, and consequently, a decrease in the efficiency of solving storage system problems.

[0034] In view of the aforementioned problems and other potential issues in traditional solutions, embodiments of this disclosure propose a technical solution for determining a target object and a set of target words associated with input text, which can improve the credibility of the target object. In embodiments of this disclosure, a computing device determines a target object associated with the text from a predetermined set of objects based on multiple weights corresponding to multiple words in the text, and also determines a set of keywords related to the determination of the target object from the multiple words. Next, within the set of keywords, the computing device determines a set of target words related to the text's topic. Then, the computing device outputs the set of target words in association with the identifier of the target object.

[0035] In embodiments of this disclosure, since the words in the target word set are both keywords in identifying the target object and words related to the text topic of the input text, the target word set output in association with the identifier of the target object can serve as useful auxiliary information to explain to the user why the target object is associated with the input text. In this way, the credibility of the target object provided by the information processing method or system in embodiments of this disclosure is improved, thereby improving the user experience of the information processing method or system. Some exemplary embodiments of this disclosure are described below with reference to the accompanying drawings.

[0036] Figure 1 A schematic diagram of an example information processing environment 100 in which embodiments of the present disclosure may be implemented is shown. For example... Figure 1 As shown, the information processing environment 100, which may also be referred to herein as the information processing system 100, may include a computing device 110 for performing information or data processing. That is, the computing device 110 can receive input information or data, process the input information or data, and provide output information or data based on the processing of the input information or data, thereby realizing a certain predetermined function or service.

[0037] For example, in Figure 1In the illustrated example, text 120 can be input as input information to computing device 110 for processing. Based on the processing of text 120, computing device 110 can determine a target object 150-T associated with text 120 from a plurality of predetermined objects 150-1 to 150-O, and output an identifier 155-T for the target object 150-T. For simplicity, the predetermined objects 150-1 to 150-O may also be collectively referred to as a plurality of predetermined objects 150 or a set of predetermined objects 150, where O is a natural number. As an example, if information processing system 100 implements a recommendation system, text 120 can be input information provided to the recommendation system, and the target object 150-T indicated by the output identifier 155-T can be a recommendation result provided by the recommendation system based on text 120.

[0038] During the processing of text 120, computing device 110 can determine multiple words 130-1 to 130-N included in text 120. For simplicity, words 130-1 to 130-N may be collectively referred to as multiple words 130 or a set of words 130, where N is a natural number. In some embodiments, words 130-1 to 130-N may be all words included in text 120, i.e., the complete set of words in text 120. In this way, the accuracy of the relevant information or processing results determined by computing device 110 based on text 120 can be improved. Alternatively, in other embodiments, words 130-1 to 130-N may also be a subset of the complete set of words in text 120, i.e., a portion of the words in text 120. Thus, the computational overhead of computing device 110 for processing text 120 can be reduced.

[0039] It should be noted that the term "words" as used herein should be broadly understood to mean any text or symbols used to express information, including not only words in the usual sense, but also any other symbols used to express or assist in expressing information. Furthermore, the text, words, or other elements in the embodiments of this disclosure are not limited to a particular language (such as Chinese or English), but are equally applicable to any known or future languages. Further, the embodiments of this disclosure can also be applied to non-human languages, such as machine programming languages, etc. Therefore, in a general sense, "text" in the embodiments of this disclosure can refer to any collection of text or symbols used to express information, while "words" in the embodiments of this disclosure can refer to an individual or combination of such text or symbols.

[0040] Continue to refer to Figure 1After determining the word set 130 in the text 120, the computing device 110 can assign corresponding weights to each word 130-1 to 130-N in the word set 130. In some embodiments, the weight of a word can represent the importance of that word in the process of determining the target object 150-T. For example, a high-weight word with a higher weight is more important in determining the target object 150-T than a low-weight word with a lower weight. That is, the computing device 110 relies more on or focuses on high-weight words when determining the target object 150-T. Therefore, in the text, the weight of a word can also be called the word's "attention" or "attention weight". Specifically, in Figure 1 In the example, word 130-1 is assigned weight 140-1, word 130-2 is assigned weight 140-2, ..., word 130-N is assigned weight 140-N, and so on. For simplicity, weights 140-1 to 140-N may be collectively referred to as multiple weights 140 or a set of weights 140 in the following text.

[0041] In some embodiments, computing device 110 may determine weights 140-1 to 140-N based on a trained text classification model for Natural Language Processing (NLP). Because NLP text classification models are typically trained on large amounts of text, they are well-versed in understanding the semantics contained in text 120, thus allowing for the reasonable assignment of corresponding weights 140-1 to 140-N to words 130-1 to 130-N in text 120, thereby improving the rationality and effectiveness of the weights 140-1 to 140-N. As an example, such a text classification model for NLP may include a text classification model based on Transformer-based Bidirectional Encoder Representation (BERT). The BERT model is a machine learning model applicable to various NLP tasks and exhibits excellent results in a wide range of NLP tasks, including text classification. A key technical feature of the BERT model is the application of bidirectional training of the attention model "Transformer" to language modeling and the fine-tuning of the language model for specific tasks. See below for reference. Figure 2 and Figure 3 To further illustrate the concept of attention weight in the embodiments of this disclosure.

[0042] Figure 2 A schematic diagram is shown illustrating how, in an embodiment of this disclosure, weights are assigned to multiple vectors to compute a result vector. Figure 2In the example, suppose the input data consists of three vectors: x1 = {0.3, -1.2, 0.4, 0.1}, x2 = {1.2, -0.6, -0.1, 0.9}, and x3 = {0.4, 0.3, 0.5, -1.3}. Furthermore, suppose these three vectors are assigned "attention" values ​​of 0.4, 0.1, and 0.5, respectively. In this case, by weighting and summing x1, x2, and x3 based on their respective attention values, we can obtain the resulting vector y = {0.4, -0.4, 0.4, -0.5} = 0.4 × {0.3, -1.2, 0.4, 0.1} + 0.1 × {0.3, -1.2, 0.4, 0.1} + 0.5 × {0.3, -1.2, 0.4, 0.1}. Therefore, in... Figure 2 In the illustrated example, the attention magnitude of each input vector is simply another way of describing the weights in the weighted averaging operation. That is, the attention or weight of each vector represents the degree of attention given to each input when calculating the weighted average.

[0043] It should be noted that, Figure 2 The application of attention weights to a weighted summation operation shown herein is merely an example and is not intended to limit the scope of this disclosure in any way. In other embodiments, the terms "weight" or "attention weight" as used herein are equally applicable to any other operation that can be weighted. Furthermore, it will be understood that Figure 2 The specific numerical values, dimensions, and number of vectors depicted herein are merely illustrative and are not intended to limit the scope of this disclosure in any way. In some embodiments, various words, text topics, predetermined themes, or other elements mentioned herein may be vectorized (also known as embedded) in an appropriate manner, thereby enabling the use of vectors similar to... Figure 2 The vectors in the vector representation can then be used to perform weighted summation or other operations that use weights.

[0044] Figure 3 A schematic diagram illustrating attention weight visualization according to embodiments of the present disclosure is shown. Figure 3 In the example depicted, it is assumed that the input content (e.g., Figure 1 The input text (120) contains two English sentences: "the rabbit quickly hooped" and "the turtle slowly crawled". For example... Figure 3As shown, the computing device 110 (or the NLP model running on it) can visualize the attention weights associated with each word. For example, the attention weight of a word is given considering its relevance to all input words (including the word itself), so these attention weights can be represented by lines between words. In some embodiments, to better process the text 120, the computing device 110 (or the NLP model running on it) can add auxiliary symbols to the input words; these auxiliary symbols will also be treated as "words" for subsequent analysis and processing. For example, Figure 3 The symbol “[SEP]” is a special separator indicating sentence boundaries, while the symbol “[CLS]” is a symbol appended to the input for classification tasks.

[0045] exist Figure 3 In the example, the lines obtained after the attention weights are visualized follow... Figure 3 The layout design connects the words being updated (left column) with the words being focused on (right column). For example, solid lines are used to represent larger attention weights, while dashed lines represent smaller attention weights. In other embodiments, the computing device 110 (or the NLP model running on it) may also use different color intensities to reflect the magnitude of attention weights. For example, weights close to 1 can be displayed as very dark lines, while weights close to 0 can be displayed as faint lines or not visible at all. Furthermore, users can highlight specific words to see attention only from that word. It should be noted that... Figure 3 For clarity, only the line showing the attention weight associated with "[CLS]" is shown as an example, while lines showing the attention weight associated with other words are omitted. However, in a real visualization, lines showing the attention weight can exist between any two words or symbols.

[0046] like Figure 3 As shown, the attention weight between two words that do not cross sentence boundaries is relatively high. This is because a trained NLP model understands that words should be associated with other words in the same sentence to best understand their context. Additionally, certain word pairs (e.g., the words "rabbit" and "hooped") can have higher attention weights than other word pairs. Figure 3 In the example, understanding the relationship between these words can help computing device 110 (or the NLP model running on it) determine that the two sentences are descriptions of a natural scene, rather than a carnivore's comment on a busy restaurant serving rabbits.

[0047] In addition, Figure 3The example also provides settings for the attention weight visualization, allowing users to configure the visualization accordingly. For instance, setting 310 allows users to select the layer number to display attention weights, setting 320 allows users to select which specific words' attention weights are associated with, and setting 330 allows users to set the color of the lines representing attention weights. Figure 3 As shown, each cell in settings bar 330 can be marked with a different color, and users can click on the corresponding colored cell to select the color of the line.

[0048] Return to reference Figure 1 In addition to outputting the identifier 155-T of the target object 150-T, the computing device 110 may also output a target word set 137 associated with the identifier 155-T. In some embodiments, the words in the output target word set 137 are both keywords that are of high importance in determining the target object 150-T and words that are related to the topic of the text 120. In other words, the target word set 137 can, on the one hand, indicate why the computing device 110 determines that the target object 150-T is associated with the text 120, and on the other hand, it can reflect the topic of the text 120. Therefore, after the user of the information processing system 100 learns about the target word set 137, he / she can fully understand and believe that the target object 150-T is associated with the text 120, thereby increasing the user's confidence in the target object 150-T. Reference will be made to... Figure 4 The computing device 110 is described in detail how it processes text 120 to derive target object 150-T and target word set 137.

[0049] It should be pointed out that, Figure 1 The information processing system 100 depicted can be applied to a wide variety of scenarios for determining associated objects based on input text. In different scenarios, the text 120 may have different content, and the multiple predetermined objects 150 may be different objects. As an example, the information processing system 100 can be used as a recommendation system in a customer service system for a storage system. In such an example, the text 120 may be descriptive text about a storage system failure, and the multiple predetermined objects 150 may be multiple predetermined knowledge bases (KBs). In some embodiments, the multiple knowledge bases may include different sets of text associated with resolving different types of failures in the storage system. For example, a particular knowledge base may contain references, helpful notes, or previous solutions for resolving a certain type of problem in the storage system, etc.

[0050] Therefore, in the aforementioned scenario where a recommendation system serves as a customer service system, based on the description text 120 of a storage system failure provided by the user, the information processing system 100 can direct technical support personnel to a target knowledge base associated with the content of the failure description text 120 from multiple predetermined knowledge bases. This guides the technical support personnel to access the target knowledge base, thereby assisting them in resolving the storage system failure. Furthermore, the information processing system 100 can also provide the technical support personnel with a target word set 137 to explain why the target knowledge base is associated with the failure description text 120, thereby increasing the technical support personnel's trust in the target knowledge base and ultimately improving the efficiency of their efforts in resolving storage system failures.

[0051] In some embodiments, computing device 110 may include any device capable of implementing computing and / or control functions, including but not limited to, a dedicated computer, a general-purpose computer, a general-purpose processor, a microprocessor, a microcontroller, or a state machine. Computing device 110 may be implemented as an individual computing device or a combination of computing devices, such as a combination of a digital signal processor (DSP) and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. It should also be noted that, in the context of this disclosure, computing device 110 may also be referred to as electronic device 110, and the two terms are used interchangeably herein.

[0052] In some embodiments, the storage device in the storage system disclosed herein may be any storage-capable device or system capable of providing storage services or functions, including but not limited to backup storage sites, cloud storage systems, hard disks (HDDs), solid-state drives (SSDs), removable disks, compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks, Blu-ray discs, serial Attached Small Computer System Interface (SCSI) disks (SAS), serial advanced technology attached (SATA) disks, any other magnetic storage devices, and any other optical storage devices, or any combination thereof.

[0053] In some embodiments, the data source or client in the storage system covered by this disclosure can refer to any device capable of generating data and receiving data storage services. In some embodiments, such devices include, but are not limited to, personal computers, tablet computers, laptop computers, notebook computers, netbook computers, any other type of computer, cellular phones or smartphones, media player devices, e-book devices, mobile WiFi devices, wearable computing devices, wireless devices, mobile devices, user devices, and any other type of electronic device.

[0054] In some embodiments, the communication links between the various components in the storage system disclosed herein can be any form of connection or coupling capable of enabling data communication or control signal communication between these components, including but not limited to coaxial cables, fiber optic cables, twisted pairs, or wireless technologies (such as infrared, radio, and microwave). In some embodiments, the communication links may also include, but are not limited to, network interface cards (NICs), hubs, modems, repeaters, bridges, switches, routers, and other devices for network connectivity, as well as various network connection lines, wireless links, etc. In some embodiments, the communication links may include various types of buses. In other embodiments, the communication links may include computer networks, communication networks, or other wired or wireless networks.

[0055] It should be understood that Figure 1 The illustrations only schematically depict units, elements, modules, or components in the information processing environment 100 that are relevant to embodiments of this disclosure. In practice, the information processing environment 100 may also include other units, elements, modules, or components for other functions. Furthermore, Figure 1 The specific number of units, elements, modules, or components shown is merely illustrative and is not intended to limit the scope of this disclosure in any way. In other embodiments, the information processing environment 100 may include any suitable number of computing devices and related elements. Therefore, embodiments of this disclosure are not limited to... Figure 1 It does not describe a specific scenario, but is generally applicable to any technological environment used for information processing.

[0056] Figure 4 A flowchart of an example information processing method 400 according to an embodiment of the present disclosure is shown. In some embodiments, method 400 may be implemented by a computing device 110 in environment 100, for example, by a processor or processing unit of computing device 110, or by various functional modules of computing device 110. In other embodiments, method 400 may also be implemented by a computing device independent of environment 100, or by other units or modules in environment 100. For ease of explanation, it will be combined with Figure 1 To describe method 400.

[0057] At box 410, computing device 110 determines a target object 150-T associated with text 120 from predetermined objects 150-1 to 150-O, based on weights 140-1 to 140-N corresponding to words 130-1 to 130-N in text 120. It should be noted that computing device 110 can determine the target object 150-T using any suitable method based on the word set 130 and the weight set 140.

[0058] For example, taking object 150-1 in object set 150 as an example, computing device 110 can vectorize object 150-1 using relevant information about object 150-1. On the other hand, computing device 110 can vectorize words 130-1 to 130-N. Then, computing device 110 can calculate N relevances between words 130-1 to 130-N and object 150-1 respectively. Next, computing device 110 can perform a weighted average of the N relevances according to the corresponding weights 140-1 to 140-N of words 130-1 to 130-N, thereby obtaining the relevance between text 120 and object 150-1. In a similar manner, computing device 110 can also calculate the relevance between text 120 and objects 150-2, ..., 150-O in object set 150, thereby obtaining O relevances between text 120 and objects 150-1 to 150-O. Subsequently, the computing device 110 can identify the object corresponding to the largest correlation among these O correlations as the target object 150-T.

[0059] In other embodiments, computing device 110 may determine target objects 150-T based on a trained text classification model for natural language processing (e.g., a BERT model). In these embodiments, the manner in which computing device 110 determines target objects 150-T from a predetermined object set 150 based on words 130 and weights 140 will depend on the specific algorithm of the text classification model. For example, a trained BERT model may vectorize words 130-1 to 130-N in the input text 120 and determine weights 140-1 to 140-N for each word 130-1 to 130-N. Then, based on the word set 130 and the weight set 140, the BERT model may determine target objects 150-T from the predetermined object set 150 according to its specific classification algorithm.

[0060] At box 410, in addition to identifying the target object 150-T, the computing device 110 also determines a set of 135 keywords related to identifying the target object 150-T within the word set 130, based on the word set 130 and the weight set 140. In other words, for finding the target object 150-T within the object set 150, the words in the keyword set 135 are "more critical" or "more important" than other words in the word set 130. That is, the keyword set 135 is a set of words that are more important in the process of identifying the target object 150-T, or a set of words that receive higher attention or focus in the process of identifying the target object 150-T.

[0061] As mentioned above, weights 140-1 to 140-N can be understood as the importance of words 130-1 to 130-N in determining the target object 150-T, or as the attention or focus of the computing device 110 on words 130-1 to 130-N when determining the target object 150-T. Therefore, the keyword set 135 in the word set 130 can be determined based on the weight set 140 corresponding to the word set 130. In some embodiments, the computing device 110 can determine words with weights higher than a weight threshold as belonging to the keyword set 135, and words with weights lower than a weight threshold as not belonging to the keyword set 135. For example, the weight threshold can be determined according to the specific technical environment and application scenario.

[0062] In another embodiment, without using a weight threshold, computing device 110 can determine the set of words with relatively large weights from words 130-1 to 130-N as the keyword set 135. For example, computing device 110 can sort the weights 140-1 to 140-N in ascending order. Then, computing device 110 can determine the set of words corresponding to the predetermined number of weights that are at the top of the weight ranking as the keyword set 135. Of course, in some examples, the keyword set 135 does not have to be the set of words at the top of the weight ranking in word set 130, but can be a set of words that are relatively high in the weight ranking. In these examples, the weight ranking of the words in keyword set 135 in word set 130 can be continuous or discontinuous.

[0063] In previous embodiments, the weights 140-1 to 140-N of words 130-1 to 130-N in text 120 were described as weight values. The weights can be directly compared. However, in other embodiments, one or more weights in the weight set 140 may not be weight values, but rather weight vectors with multiple weight components. Weight vectors cannot be directly compared to weight values ​​or other weight vectors. For example, suppose the weight 140-1 of word 130-1 is not a weight value, but a weight vector composed of multiple weight components. In this case, the measurement of the weight 140-1 may need to be determined based on the individual weight components of weight 140-1, and consequently, the keyword set 135 in word set 130 also needs to consider the individual weight components of the weights. References will follow later. Figures 5 to 7 To describe such an embodiment.

[0064] At box 420, after determining the keyword set 135 within the word set 130, the computing device 110 further determines a target word set 137 related to the text topic 125 of the text 120 within the keyword set 135. That is, within the keyword set 135, some words can be considered related to the text topic 125 and belong to the target word set 137, while other words can be considered unrelated to the text topic 125 and do not belong to the target word set 137. It should be noted that in embodiments of this disclosure, words not included in the target word set 137 may not be completely unrelated to the text topic 125, but rather have a low relevance to the text topic 125, for example, below a predetermined relevance threshold.

[0065] In some embodiments, in order to determine the target word set 137 in the keyword set 135, the computing device 110 may first determine the text topic 125 of the text 120. For example, the computing device 110 may analyze the content recorded in the text 120 to summarize or generalize the text topic 125. After determining the text topic 125, the computing device 110 may derive the target word set 137 based on the relevance between each word in the keyword set 135 and the text topic 125.

[0066] In some cases, the text topic 125 of text 120 can be represented using one or more keyword terms. Keyword terms can be words in text 120 or other words not included in text 120. When keyword terms are used to represent the text topic 125, the relevance between a word in the keyword set 135 and the text topic 125 can be represented as the relevance between that word and these keyword terms. As a simple example, suppose the text topic 125 can be represented by the keyword terms "read," "data," and "timeout." In this case, the computing device 110 can determine the set of words in the keyword set 135 that are semantically close to these keyword terms as the target word set 137.

[0067] In other embodiments, as a quantitative calculation method for relevance, computing device 110 can vectorize each word in word set 130 and vectorize one or more topic words of text topic 125. Then, computing device 110 can calculate the sum of relevance between the vectors of each word in word set 130 and the topic word vectors of text topic 125. Next, computing device 110 can determine words whose sum of relevance is higher than a relevance threshold as relevant to text topic 125, and words whose sum of relevance to text topic 125 is lower than the relevance threshold as irrelevant to text topic 125.

[0068] In other embodiments, without using a relevance threshold, the computing device 110 can determine the set of words in the keyword set 135 that have a relatively high relevance to the text topic 125 as the target word set 137. For example, the computing device 110 can sort the relevance between each word in the keyword set 135 and the text topic 125 in ascending order. Then, the computing device 110 can determine the set of words corresponding to the predetermined number of relevance words that are at the top or relatively high in the order of relevance as the target word set 137. (Referring to the following...) Figure 8 and Figure 9 To describe such an embodiment.

[0069] At box 430, after determining the target object 150-T and the target word set 137, the computing device 110 outputs the target word set 137 in association with the identifier 155-T of the target object 150-T. That is, the computing device 110 can output both the identifier 155-T and the target word set 137 in an associated manner so that the user of the information processing system 100 understands that the target word set 137 is auxiliary information associated with the target object 150-T indicated by the identifier 155-T.

[0070] In some embodiments, the association between the output identifier 155-T and the target word set 137 can be expressed explicitly. For example, the computing device 110 can indicate the association between the identifier 155-T and the target word set 137 using explicit text or symbols. As an illustrative example, the computing device 110 could output: "The target object 150-T with identifier 155-T is associated with text 120, and the target word set 137 is supplementary information about this," or similar explicit statements, etc. In other embodiments, the computing device 110 can also implicitly represent the association between the output identifier 155-T and the target word set 137. For example, the computing device 110 can output both the identifier 155-T and the target word set 137 at two adjacent locations on a display device, thereby implicitly reflecting the association between the two.

[0071] In some embodiments, the computing device 110 may output the target word set 137 as explanatory information about the association between text 120 and target object 155-T. In other words, the computing device 110 may output the target word set 137 to explain why target object 150-T is identified as associated with input text 120 among a plurality of predetermined objects 150. In this way, the user of the information processing system 100 can not only know the target object 150-T associated with text 120, but also understand why target object 150-T is associated with text 120, thereby increasing the user's confidence in target object 150-T and enhancing the user experience and efficiency of using the information processing system 100. In some embodiments, the information processing system 100 may implement a recommendation system, in which case target object 150-T may be the recommendation result of the recommendation system, and target word set 137 may be the reason why target object 150-T is recommended.

[0072] Through information processing method 400, the information processing system 100 (e.g., a recommendation system) implemented by computing device 110 can not only indicate the target object 150-T as the information processing result (e.g., a recommendation result) to the user, but also provide the user with a set of target words 137 as auxiliary information to explain to the user why the target object 150-T is associated with the input text 120. Therefore, the user's confidence in the information processing result (e.g., the recommendation result) of the information processing system 100 is improved, thereby enhancing the user experience and efficiency when using the information processing system.

[0073] As mentioned above in the description of example method 400, in some embodiments, one or more weights in the weight set 140 corresponding to the word set 130 may be a weight vector comprising multiple weight components. In these embodiments, the magnitude measurement of the weights may need to take into account the individual weight components of the weights. Therefore, the determination of the keyword set 135 in the word set 130 also needs to be based on the weight components of the weight vector in the weight set 140. See below for reference. Figures 5 to 7 To describe such an embodiment.

[0074] Figure 5 A flowchart illustrating an example process 500 for determining a keyword set 135 according to an embodiment of the present disclosure is shown. In some embodiments, process 500 may be implemented by a computing device 110 in environment 100, for example, by a processor or processing unit of computing device 110, or by various functional modules of computing device 110. In other embodiments, process 500 may also be implemented by a computing device independent of environment 100, or by other units or modules in environment 100.

[0075] Figure 6 A schematic diagram is shown illustrating the determination of weighted average values ​​610-1, 610-2, ..., 610-N based on multiple components with weights 140-1, 140-2, ..., 140-N according to embodiments of the present disclosure. It should be noted that, although in Figure 6 In the example, each weight in the weight set 140 is a weight vector comprising multiple weight components; however, this is merely illustrative and is not intended to limit the scope of this disclosure in any way. In other embodiments, a portion of the weights in the weight set 140 may be weight vectors comprising multiple weight components, while another portion of the weights may be weight values. Figure 7 A schematic diagram is shown illustrating the determination of a keyword set 135 based on the weighted average values ​​610-1 to 610-N of words 130-1 to 130-N according to an embodiment of the present disclosure. Hereinafter, for simplicity of description, the weighted average values ​​610-1 to 610-N may also be referred to as a plurality of weighted average values ​​610 or a set of weighted average values ​​610.

[0076] First refer to Figure 6 In the illustrated example, each weight in the weight set 140 corresponding to the word set 130 can be a weight vector comprising multiple weight components. Taking word 130-1 as an example, the corresponding weight 140-1 is a weight vector {a}. 11 ,a 12 ,…,a 1N}, where the weight component a 11 This represents the weight determined based on the relevance of word 130-1 to itself, with weight component a. 12 This represents the weights determined based on the relevance between word 130-1 and word 130-2, ..., while a 1N This represents the weight determined based on the relevance between word 130-1 and word 130-N. In other words, the weight component a of weight 140-1. 11 a 12 ... a 1N It is determined based on the correlation between words 130-1 to 130-N and word 130-1. This method of determining attention weights can also be called a "self-attention" mechanism. That is to say, in Figure 6 In the example, computing device 110 employs a self-attention mechanism to assign attention weights 140-1 to 140-N to words 130-1 to 130-N in text 120, thereby improving the rationality of weight set 140.

[0077] Similarly, the weight 140-2 corresponding to word 130-2 can be represented as a vector {a 21 ,a 22 ,…,a 2N}, where the weight component a 21 The weights are determined based on the relevance between word 130-2 and word 130-1, with weight component a. 22 This indicates the weights determined based on the relevance of word 130-2 to itself, ..., while a 2N This represents the weight determined based on the relevance between word 130-2 and word 130-N. Similarly, the weight 140-N corresponding to word 130-N can be represented as a vector {a}. N1 ,a N2 ,…,a NN}, where the weight component a N1 The weights are determined based on the relevance between words 130-N and 130-1, with weight component a. N2 This represents the weights determined based on the relevance between terms 130-N and 130-2, ..., while a NN This represents the weights determined based on the relevance of words 130-N to themselves. In some embodiments, the weight vector {a} 11 ,a 12 ,…,a 1N}、{a 21 ,a 22 ,…,a 2N}、……、{a N1 ,a N2 ,…,a NN The matrix formed by these matrixes can be called the weight matrix or attention matrix. Based on this weight matrix, similar attention-based algorithms can be generated. Figure 3 The visual graphics depicted in the text.

[0078] Also refer to Figure 5 and Figure 6 ,exist Figure 5 At frame 510, computing device 110 determines the average weights 610-1 to 610-N corresponding to weights 140-1 to 140-N, i.e., the set of average weights 610. For example... Figure 6 As shown, each weight average in the set of weight averages 610 is determined based on multiple weight components of the corresponding weight. For example, the weight average 610-1 corresponding to word 130-1 is weight component a of weight 140-1. 11 a 12 ... a 1N The average value. Similarly, the average weight 610-2 corresponding to word 130-2 is the weight component a of weight 140-2. 21 a 22 ... a 2N The average value of the weights, and the average weight of the word 130-N, 610-N, is the weight component a of the weight of the word 140-N.N1 a N2 ... a NN The average value. That is, each weight average value in the weight average value set 610 is the average value calculated column-wise in the weight matrix of the weight set 140.

[0079] It should be noted that, although in Figure 6 In the example, the average of the weight components of the weight vector is used as a measure of the weight magnitude of the weight vector; however, this is merely illustrative and not intended to limit the scope of this disclosure in any way. In other embodiments, the computing device 110 may also employ other calculation methods to measure the weight magnitude of the weight vector based on its weight components. For example, the computing device 110 may calculate the sum or normalized sum of the individual weight components of the weight vector to measure its weight magnitude.

[0080] refer to Figure 5 and Figure 7 ,exist Figure 5 At box 520, after determining the set of average weights 610 corresponding to the set of weights 140, the computing device 110 determines a predetermined number (K, K≤N, K is a natural number) of average weights 610-X, 610-Y, ..., 610-W in order of size from the average weights 610-1 to 610-N. For simplicity, this predetermined number K can also be referred to as the first number K in the following text. Figure 7 As depicted, in some embodiments, computing device 110 may first sort the weight averages 610-1 to 610-N by size to obtain an ordered set 710 of weight averages. That is, the ordered set 710 of weight averages still includes N weight averages, but the N weight averages are arranged in ascending order. For example, weight average 610-X is the largest weight average in the set 610, weight average 610-Y is the next largest, and so on, while weight average 610-Z is the smallest weight average in the set 610.

[0081] Then, the computing device 110 can determine an ordered subset 720 of the weighted averages from the ordered set 710 of the weighted averages based on the order of the weighted averages. Figure 7In the illustrated example, the ordered subset 720 of weighted averages includes the first K weighted averages that are ranked first in size within the ordered set 710 of weighted averages, namely, weighted averages 610-X, 610-Y, ..., 610-W. In other embodiments, the ordered subset 720 of weighted averages may also be the first K weighted averages that are ranked relatively first in size within the ordered set 710 of weighted averages, i.e., the K relatively large weighted averages. Furthermore, in such embodiments, the first K weighted averages in the ordered subset 720 need not be ranked consecutively within the ordered set 710 of weighted averages.

[0082] Continue to refer to Figure 5 and Figure 7 ,exist Figure 5 At frame 530, after determining the ordered subset 720 of the average weights, the computing device 110 determines from the words 130-1 to 130-N the first number of K words 130-X, 130-Y, ..., 130-W corresponding to the first number of K average weights 610-X, 610-Y, ..., 610-W, as... Figure 1 The set of keywords described in the text is 135. This is because, in Figures 5 to 7 In the example described, since the words 130-X, 130-Y, ..., 130-W correspond to larger average weights 610-X, 610-Y, ..., 610-W respectively, that is, these words are more important or receive more attention from the computing device 110 in the process of determining the target object 150-T, the computing device 110 can determine the first number K words 130-X, 130-Y, ..., 130-W as the keyword set 135.

[0083] Through example process 500, even if one or more weights in the weight set 140 corresponding to the word set 130 are weight vectors and cannot be directly compared with weight values ​​or other weight vectors, the computing device 110 can reasonably measure the weight of these weight vectors and achieve a reasonable comparison with the weight values ​​or other weight vectors, thereby improving the rationality of the target object 150-T and the target word set 137, and thus enhancing the performance of the information processing system 100.

[0084] As mentioned above in the description of example method 400, in some embodiments, computing device 110 may sort the relevance between each word in keyword set 135 and text topic 125 in ascending order. Then, computing device 110 may determine the set of words corresponding to the predetermined number of most or relatively high-ranking relevance words as the target word set 137. See below for reference. Figure 8 and Figure 9 To describe such an embodiment.

[0085] Figure 8 A flowchart illustrating an example process 800 for determining a target word set 137 according to an embodiment of the present disclosure is shown. In some embodiments, process 800 may be implemented by a computing device 110 in environment 100, for example, by a processor or processing unit of computing device 110, or by various functional modules of computing device 110. In other embodiments, process 800 may also be implemented by a computing device independent of environment 100, or by other units or modules in environment 100.

[0086] Figure 9 A schematic diagram is shown illustrating how a target word set 137 is determined based on text topic relevance 910-X to 910-W between a plurality of words 130-X to 130-W and text topic 125, according to an embodiment of the present disclosure. For simplicity of description, Figure 9 China will continue to use Figure 7 Some reference numerals appear in the text. However, it should be noted that in... Figure 7 and Figure 9 The use of reference numbers by the CPC does not mean Figure 9 The embodiments require Figure 7 Based on the embodiments. Specifically, Figure 7 Implementation examples and Figure 9 The embodiments can be implemented independently of each other or in combination. For example, Figure 9 The set of keywords 135 described in the text (e.g., including K words 130-X, 130-Y, ..., 130-W) can be as follows: Figure 7 It can be derived by comparing the average values ​​of the weighted components of the weight vector, as in the example, or by directly comparing the weight values.

[0087] refer to Figure 8 and Figure 9 ,exist Figure 8At frame 810, computing device 110 determines the first number K words 130-X, 130-Y, ..., 130-W in keyword set 135 and their corresponding first number K text topic relevances 910-X, 910-Y, ..., 910-W to text topic 125. For simplicity, these text topic relevances 910-X, 910-Y, ..., 910-W may also be referred to as multiple text topic relevances 910 or a set of text topic relevances 910. Specifically, text topic relevance 910-X is the relevance between word 130-X and text topic 125, text topic relevance 910-Y is the relevance between word 130-Y and text topic 125, ..., and text topic relevance 910-W is the relevance between word 130-W and text topic 125.

[0088] It will be understood that computing device 110 can determine the relevance between a word (e.g., word 130-X) and text topic 125 using any suitable method. For example, computing device 110 can vectorize word 130-X into word vectors and text topic 125 into text topic vectors. Then, computing device 110 can determine the text topic relevance 910-X between word 130-X and text topic 125 by calculating the similarity or distance between the word vector of word 130-X and the text topic vector of text topic 125. To this end, computing device 110 can use any suitable similarity or distance between vectors to calculate the text topic relevance 910-X, including but not limited to Euclidean distance, Minkowski distance, Manhattan distance, Chebyshev distance, Hamming distance, Mahalanobis distance, cosine similarity, adjusted cosine similarity, Pearson correlation coefficient, Jaccard similarity coefficient, etc.

[0089] Furthermore, it should be noted that the computing device 110 can vectorize the text topic 125 in any suitable manner. For example, the computing device 110 can use one or more topic terms to represent the text topic 125. Then, the computing device 110 can derive the text topic vector of the text topic 125 through appropriate operations (e.g., averaging) between the word vectors corresponding to the one or more topic terms. In other embodiments, the text topic 125 can be represented as a topic distribution about multiple predetermined topics, that is, the text topic 125 is a probability distribution of multiple predetermined topics. For example, the computing device 110 can use a topic model to determine the topic distribution of the text topic 125. In such an embodiment, the computing device 110 can determine the relevance between the word 130-X and the text topic 125 through the relevance between the word 130-X and multiple predetermined topics and the probability distribution of the predetermined topics. Reference will be made later. Figures 10 to 13 To describe such an embodiment.

[0090] Continue to refer to Figure 8 and Figure 9 ,exist Figure 8 At box 820, after determining the text topic relevance set 910, the computing device 110 determines a predetermined number (L, L≤K, L is a natural number) of text topic relevances 910-U, 910-V, ..., 910-T from the first number K text topic relevances 910-X, 910-Y, ..., 910-W in the text topic relevance set 910, in ascending order. In the following text, this predetermined number L can also be referred to as the second number L. For example, in... Figure 9 In the example, computing device 110 can first sort the text topic relevance 910-X, 910-Y, ..., 910-W according to their relevance magnitude, thereby obtaining an ordered set 920 of text topic relevance. The ordered set 920 of text topic relevance still includes a first number of K text topic relevances, but the K text topic relevances are arranged in order of magnitude. That is, text topic relevance 910-U is the largest text topic relevance in the set 910, followed by text topic relevance 910-V, ..., while text topic relevance 910-S is the smallest text topic relevance in the set 910.

[0091] Then, the computing device 110 can determine an ordered subset 930 of text topic relevance from the ordered set 920 of text topic relevance, namely, text topic relevances 910-U, 910-V, ..., 910-T, for a total of L text topic relevances. Figure 9 In the illustrated example, the ordered subset 930 of text topic relevance includes the second-largest L text topic relevances in the ordered set 920 of text topic relevances, i.e., text topic relevances 910-U, 910-V, ..., 910-T. In other embodiments, the ordered subset 930 of text topic relevances may also be the second-largest L text topic relevances in the ordered set 920 of text topic relevances, i.e., the larger L text topic relevances. Furthermore, in such embodiments, the second-largest L text topic relevances in the ordered subset 930 need not be ordered consecutively in the ordered set 920 of text topic relevances.

[0092] Continue to refer to Figure 8 and Figure 9 ,exist Figure 8At frame 830, after determining the ordered subset 930 of text topic relevance, the computing device 110 determines the set of words corresponding to the ordered subset 930 of text topic relevance from the first number K words 130-X, 130-Y, ..., 130-W of the keyword set 135, that is, the second number L words 130-U, 130-V, ..., 130-T corresponding to the second number L text topic relevances 910-U, 910-V, ..., 910-T, as... Figure 1 The target word set described in the text is 137. This is because, in Figure 8 and Figure 9 In the depicted embodiment, since words 130-U, 130-V, ..., 130-T correspond to greater text topic relevance 910-U, 910-V, ..., 910-T respectively, that is, in the keyword set 135, these words are more relevant to the text topic 125 of text 120 compared to other words, the computing device 110 can determine the second number L words 130-U, 130-V, ..., 130-T in the keyword set 135 as... Figure 1 The target word set described in the text is 137.

[0093] Through the example process 800, the computing device 110 can quantitatively calculate the text topic relevance between each word in the keyword set 135 and the text topic 125, and then determine the target word set 137 in the keyword set 135 that has a greater relevance to the text topic 125, thereby improving the rationality of the target word set 137 and thus enhancing the performance of the information processing system 100.

[0094] As mentioned above in the description of example process 800, in some embodiments, text topic 125 can be represented as a topic distribution about multiple predetermined topics, that is, text topic 125 is a probability distribution of multiple predetermined topics. For example, computing device 110 can use a topic model to determine the topic distribution of text topic 125. In such an embodiment, computing device 110 can determine the relevance between word 130-X and text topic 125 by the relevance between word 130-X and multiple predetermined topics and the probability distribution of the predetermined topics. Reference will be made below. Figures 10 to 13 To describe such an embodiment.

[0095] Figure 10A schematic diagram is shown illustrating the use of topic model 1010 to determine text topic 1030 of text 1020 according to an embodiment of the present disclosure. Generally, topic modeling or topic modeling is an unsupervised machine learning technique that scans a set of documents, detects word and phrase patterns in the documents, and automatically clusters word groups and similar expressions that characterize the features of that set of documents. For example, Latent Dirichlet Allocation (LDA) is an example of a topic model that can be used to classify documents or text into specific topics. Specifically, an LDA model can construct a per-document model of topics and a per-topic model of words, and model it as a Dirichlet distribution. It should be noted that although topic models are described using LDA models as examples in some embodiments, this is merely illustrative and is not intended to limit the scope of the present disclosure in any way. In other embodiments, the topic model herein can be any known or future-developed topic model.

[0096] exist Figure 10 In the example, topic model 1010 includes three predefined topics: topic #1, topic #2, and topic #3. Each predefined topic is a word distribution, that is, the probability distribution of the occurrence of each word under the predefined topic. Figure 10 The expression is represented as p*word, where p represents the probability of a word appearing. For example... Figure 10 As further illustrated, suppose the topic distribution of text 1020 regarding three predetermined topics is as follows: the probability of topic #1 is 50%, the probability of topic #2 is 30%, and the probability of topic #3 is 20%. In this case, using topic model 1010 to perform topic analysis and statistics on text 1020 will yield a topic distribution (i.e., result) 1030. For example... Figure 10 As shown, result 1030 marks each word in text 1020 as belonging to a specific topic. For example, "word#1" represents a word belonging to topic #1, "word#2" represents a word belonging to topic #2, and "word#3" represents a word belonging to topic #3. In other embodiments, words belonging to different topics in result 1030 can also be distinguished by different colors. Furthermore, result 1030 can also use bar icons of varying lengths to visually represent the relative probabilities of different topics.

[0097] After explaining the basic concepts and principles of topic models, the following will refer to Figures 11 to 13The following describes how computing device 110 determines the text topic relevance 910-X between word 130-X and text topic 125 based on the topic distribution of text topic 125. It should be noted that in the following description, word 130-X will be used as an example to illustrate how computing device 110 derives the text topic relevance 910-X corresponding to word 130-X. However, it will be understood that for other words 130-Y, ..., 130-W in the keyword set 135, computing device 110 can use a similar method to determine the text topic relevance 910-Y, ..., 910-W corresponding to words 130-Y, ..., 130-W, respectively.

[0098] Figure 11 A flowchart illustrating an example process 1100 for determining text topic relevance 910-X between words 130-X in a keyword set 135 and text topic 125, according to embodiments of the present disclosure, is shown. In some embodiments, process 1100 may be implemented by a computing device 110 in environment 100, for example, by a processor or processing unit of computing device 110, or by various functional modules of computing device 110. In other embodiments, process 1100 may also be implemented by a computing device independent of environment 100, or by other units or modules in environment 100.

[0099] Figure 12 A schematic diagram is shown illustrating the representation of text topics 125 as a topic distribution with respect to predetermined topics 1210-1 to 1210-M according to an embodiment of the present disclosure. Hereinafter, for the sake of simplicity, predetermined topics 1210-1 to 1210-M may also be collectively referred to as a plurality of predetermined topics 1210 or a set of predetermined topics 1210, where M is a natural number. Figure 13 A schematic diagram is shown illustrating the determination of text topic relevance 910-X based on predetermined topic relevance 1310-1 to 1310-M between words 130-X and predetermined topics 1210-1 to 1210-M according to an embodiment of the present disclosure. Hereinafter, for simplicity of description, predetermined topic relevances 1310-1 to 1310-M may also be collectively referred to as a plurality of predetermined topic relevances 1310 or a set of predetermined topic relevances 1310.

[0100] refer to Figure 11 and Figure 12 ,exist Figure 11 At box 1110, computing device 110 determines that the text topics 125 are probabilities 1215-1 to 1215-M of predetermined topics 1210-1 to 1210-M. For simplicity, probabilities 1215-1 to 1215-M may also be collectively referred to as multiple probabilities 1215 or probability set 1215. Therefore, in Figure 12In the illustrated example, the text topic 125 of text 120 is represented by a topic distribution with respect to predetermined topics 1210-1 to 1210-M. Specifically, text topic 125 can be represented as predetermined topic 1210-1 and its corresponding probability 1215-1, predetermined topic 1210-2 and its corresponding probability 1215-2, ..., predetermined topic 1210-M and its corresponding probability 1215-M. For example, the topic distribution of text topic 125 with respect to the predetermined topic set 1210 can be understood as text topic 125 having a probability of 1215-1 for predetermined topic 1210-1, a probability of 1215-2 for predetermined topic 1210-2, ..., and a probability of 1215-M for predetermined topic 1210-M.

[0101] In some embodiments, computing device 110 may use a topic model (e.g., an LDA model) to determine the topic distribution of text topic 125 with respect to a predetermined topic set 1210. That is, computing device 110 may determine the probability set 1215 corresponding to the predetermined topic set 1210 based on the topic model used to determine text topic 125. To make the topic model more suitable for the technical scenarios of the embodiments of this disclosure, the topic model used to determine text topic 125 may be trained using a text set associated with multiple predetermined objects 150. In this way, the topic model used to determine text topic 125 can learn the topic and word distributions involved in the multiple predetermined objects 150, thereby improving the effectiveness of the topic model in analyzing and statistically analyzing text topic 125 of text 120.

[0102] refer to Figure 11 and Figure 13 ,exist Figure 11 At frame 1120, computing device 110 determines the predetermined topic relevance 1310-1 to 1310-M between word 130-X, 130-Y, ..., 130-W and predetermined topics 1210-1 to 1210-M, i.e., the predetermined topic relevance set 1310. Specifically, predetermined topic relevance 1310-1 is the relevance between word 130-X and predetermined topic 1210-1, predetermined topic relevance 1310-2 is the relevance between word 130-X and predetermined topic 1210-2, ..., and predetermined topic relevance 1310-M is the relevance between word 130-X and predetermined topic 1210-M.

[0103] It will be understood that computing device 110 can determine the relevance (e.g., pre-defined topic relevance 1310-1) between word 130-X and a predetermined topic (e.g., predetermined topic 1210-1) using any suitable method. As an example, computing device 110 can vectorize word 130-X into word vectors and predetermined topic 1210-1 into predetermined topic vectors. Then, computing device 110 can calculate the pre-defined topic relevance 1310-1 between word 130-X and predetermined topic 1210-1 by calculating the similarity or distance between the word vector of word 130-X and the predetermined topic vector of predetermined topic 1210-1. For this purpose, computing device 110 can use any suitable similarity or distance between vectors to calculate the pre-defined topic relevance 1310-1, including but not limited to Euclidean distance, Minkowski distance, Manhattan distance, Chebyshev distance, Hamming distance, Mahalanobis distance, cosine similarity, adjusted cosine similarity, Pearson correlation coefficient, Jaccard similarity coefficient, etc.

[0104] Furthermore, it should be noted that the computing device 110 can vectorize the predetermined topic 1210-1 in any suitable manner. For example, the computing device 110 can use one or more topic terms to represent the predetermined topic 1210-1. Then, the computing device 110 can derive the predetermined topic vector of the predetermined topic 1210-1 through appropriate operations (e.g., averaging) between the word vectors corresponding to the one or more topic terms. In other embodiments, such as when using a topic model to determine the topic distribution of text topic 125, the predetermined topic 1210-1 can be represented as the distribution of multiple words, that is, the probability distribution of word occurrences under the predetermined topic 1210-1. In such an embodiment, the computing device 110 can determine the correlation between word 130-X and the predetermined topic 1210-1 through the correlation between word 130-X and multiple words of the predetermined topic 1210-1 and the probability distribution of these words. Reference will be made later. Figures 14 to 16 To describe such an embodiment.

[0105] Continue to refer to Figure 11 and Figure 13 ,exist Figure 11At box 1130, the computing device 110 determines the text topic relevance 910-X corresponding to the word 130-X based on predetermined topic relevances 1310-1 to 1310-M and probabilities 1215-1 to 1215-M. For example, since the text topics 125 have different probabilities 1215 for multiple predetermined topics 1210, in order to reflect the different probability magnitudes of each predetermined topic 1210-1 to 1210-M in the text topic relevance 910-X, the computing device 110 can perform a weighted summation of the predetermined topic relevances 1310-1 to 1310-M according to the probabilities 1215-1 to 1215-M of each predetermined topic 1210-1 to 1201-M, thereby obtaining the text topic relevance 910-X. It should be noted that the weighted summation operation described herein is only an example of how to calculate the text topic relevance 910-X and is not intended to limit the scope of this disclosure in any way. In other embodiments, other calculation methods are possible using the predetermined topic relevance set 1310 and probability set 1215. For example, the computing device 110 may first exclude predetermined topic relevances with probabilities below a threshold, and then perform a weighted summation of the remaining predetermined topic relevances, and so on.

[0106] Through the example process 1100, the computing device 110 can determine the text topic relevance 910-X between words 130-X in the keyword set 135 and the text topic 125 based on the topic distribution of the text topic 125 with respect to the predetermined topic set 1210. This can improve the accuracy of the text topic relevance 910-X and thus improve the rationality of the final output target word set 137.

[0107] As mentioned above in the description of example process 1100, in some embodiments, computing device 110 can determine the relevance between word 130-X and predetermined topic 1210-1 by the relevance between word 130-X and multiple words of predetermined topic 1210-1 and the probability distribution of these words. See below for reference. Figures 14 to 16 This embodiment will be described below. It should be noted that, in the following description, a predetermined topic 1210-1 will be used as an example to illustrate how the computing device 110 derives the predetermined topic correlation 1310-1 corresponding to predetermined topic 1210-1. However, it will be understood that for other predetermined topics 1210-2, ..., 1210-M in the predetermined topic set 1210, the computing device 110 can use a similar method to determine the predetermined topic correlations 1310-2, ..., 1310-M corresponding to predetermined topics 1210-2, ..., 1210-M, respectively.

[0108] Figure 14A flowchart illustrating an example process 1400 for determining a predetermined topic relevance 1310-1 between words 130-X in a keyword set 135 and a predetermined topic 1210-1, according to embodiments of the present disclosure, is shown. In some embodiments, process 1400 may be implemented by a computing device 110 in environment 100, for example, by a processor or processing unit of computing device 110, or by various functional modules of computing device 110. In other embodiments, process 1400 may also be implemented by a computing device independent of environment 100, or by other units or modules within environment 100. Figure 15 A schematic diagram is shown of a predetermined topic 1210-1 including a word set 1510 and a probability set 1515, and a word vector set 1520 corresponding to the word set 1510, according to an embodiment of the present disclosure. Figure 16 A schematic diagram is shown illustrating how a predetermined topic relevance 1310-1 is determined based on a predetermined topic vector 1610-1 and a word vector 1620-X, according to an embodiment of the present disclosure.

[0109] refer to Figure 14 and Figure 15 ,exist Figure 14 At frame 1410, computing device 110 determines the word set 1510 and the corresponding probability set 1515 included in predetermined topic 1210-1 to 1210-M. That is, in Figure 15 In the illustrated example, the predetermined topic 1210-1 is represented by a word distribution. Specifically, the predetermined topic 1210-1 can be represented as word 1510-1 and its corresponding probability 1515-1, word 1510-2 and its corresponding probability 1515-2, ..., word 1510-P and its corresponding probability 1515-P, where P is a natural number. It should be noted that the words in the word set 1510 associated with the predetermined topic 1210-1 are words used to represent the predetermined topic 1210-1, determined, for example, through the training process of the topic model, and do not have a specific association with the word set 130 in the text 120. The distribution of the predetermined topic 1210-1 with respect to the word set 1510 can be understood as follows: the probability of word 1510-1 appearing under the predetermined topic 1210-1 is 1515-1, the probability of word 1510-2 appearing is 1515-2, ..., and the probability of word 1510-P appearing is 1515-P. In some embodiments, computing device 110 may use a topic model (e.g., an LDA model) to determine a word set 1510 and a probability set 1515 for a predetermined topic 1210-1.

[0110] Continue to refer to Figure 14 and Figure 15 ,exist Figure 14At frame 1420, computing device 110 determines word vector set 1520 of word set 1510. Specifically, computing device 110 can determine word vector 1520-1 for word 1510-1, word vector 1520-2 for word 1510-2, ..., and word vector 1520-P for word 1510-P. It will be understood that computing device 110 can use any suitable method to vectorize (also called word embedding) words 1510-1 to 1510-P into corresponding word vectors 1520-1 to 1520-P, for example, using artificial neural networks, reducing the dimensionality of word co-occurrence matrices, using probabilistic models, and using explicit representations based on the context of the words, etc. In some embodiments, reference... Figure 1 When text 120 is input into computing device 110, computing device 110 can perform word embedding on word set 130 to vectorize words 130-1 to 130-N. In these embodiments, computing device 110 can vectorize word set 1510 in the same way as vectorizing word set 130, for example, by using a BERT model to perform word embedding.

[0111] refer to Figure 14 and Figure 16 ,exist Figure 14 At frame 1430, after determining the word vector set 1520, the computing device 110 determines the predetermined topic vector 1610-1 for the predetermined topic 1210-1 based on the probability set 1515 and the word vector set 1520. For example, since the probabilities 1515-1 to 1510-P of words 1510-1 to 1510-P appearing in the predetermined topic 1210-1 are different, in order to reflect the different probabilities of each word 1510-1 to 1510-P in the predetermined topic vector 1610-1, the computing device 110 can perform a weighted summation of word vectors 1520-1 to 1520-P according to the probabilities 1515-1 to 1515-P of words 1510-1 to 1510-P, thereby obtaining the predetermined topic vector 1610-1 used to represent the predetermined topic 1210-1. It should be noted that the weighted summation operation described herein is merely an example of how to calculate the predetermined topic vector 1610-1, and is not intended to limit the scope of this disclosure in any way. In other embodiments, other calculation methods using the word vector set 1520 and the probability set 1515 are possible. For example, the computing device 110 may first exclude word vectors with probabilities below a threshold, and then perform a weighted summation on the remaining word vectors, and so on.

[0112] Continue to refer to Figure 14 and Figure 16 ,exist Figure 14At box 1440, after determining the predetermined topic vector 1610-1 of the predetermined topic 1210-1, the computing device 110 determines the predetermined topic relevance 1310-1 corresponding to the predetermined topic 1210-1 based on the word vector 1620-X of the word 130-X and the predetermined topic vector 1610-1. It will be understood that the computing device 110 can use any suitable method to vectorize the word 130-X into the corresponding word vector 1620-X. In some embodiments, reference... Figure 1 When text 120 is input into computing device 110, computing device 110 can perform word embedding on words 130-X to vectorize words 130-X, so as to facilitate subsequent processing of text 120. In these embodiments, computing device 110 can directly use the previous embedding results of words 130-X as... Figure 16 The word vector 1620-X in the text. Furthermore, it will be understood that the computing device 110 can use any appropriate similarity or distance between vectors to calculate the predetermined topic relevance 1310-1, including but not limited to Euclidean distance, Minkowski distance, Manhattan distance, Chebyshev distance, Hamming distance, Mahalanobis distance, cosine similarity, adjusted cosine similarity, Pearson correlation coefficient, Jaccard similarity coefficient, etc.

[0113] Through the example process 1400, the computing device 110 can determine the predetermined topic relevance 1310-1 between words 130-X in the keyword set 135 and the predetermined topic 1210-1 based on the probability set 1515 and the word vector set 1520 corresponding to the word set 1510 of the predetermined topic 1210-1, thereby improving the accuracy of the predetermined topic relevance 1310-1 and thus improving the rationality of the final output target word set 137.

[0114] The foregoing has described the general principles and several exemplary embodiments of the technical solutions disclosed herein. As mentioned earlier, the embodiments of this disclosure can be applied to recommendation systems. The following section will use the application of the technical solutions of this disclosure to a service request (SR) recommendation system as an example to specifically illustrate an exemplary application of the technical solutions of this disclosure.

[0115] In companies delivering large-scale and complex information services, technical support engineering teams handle a large volume of customer service requests. These requests may include reports of problems caused by software defects, hardware or network issues, or operational errors. Finding appropriate knowledge references that can resolve these service requests is a valuable task in the service request classification process.

[0116] In most customer service systems of information service providers, customers can submit service requests about problems encountered while using the information system through a "Customer Service Request Tracking and Resolution Module," including, for example, a title (summary) and a detailed description. Automated service request classification algorithms can be formulated as a classification problem, using the title and description as input features and mapping them to one of the available knowledge references (classifications). The main challenge is that service requests often contain a combination of free, unstructured text, such as customer descriptions, system logs, memory dumps, performance statistics, and stack traces, which can make the input data noisy.

[0117] When providing technical support, some junior engineers may lack experience and therefore require the relevant experience and assistance of senior engineers (e.g., through annotations, previous solutions, etc.). In this regard, the SR (Recommended Knowledge Base) system can be used to recommend relevant knowledge bases (KBs) containing annotations or previous solutions useful for resolving the issues in the current service request. Therefore, the business objective of the SR system is to facilitate the process of classifying customer service requests by building a machine learning model. This machine learning model can recommend appropriate knowledge references to the technical support engineers handling the customer service requests, thereby improving the efficiency of junior technical support engineers based on the recommended knowledge references. However, some junior engineers may not trust the recommendation system's results if the system does not explain why these knowledge bases are recommended to them. Therefore, a reasonable explanation of the recommendation results becomes important.

[0118] The machine learning models used in SR recommender systems are generally based on Natural Language Processing (NLP) models (e.g., BERT). Many works have attempted to interpret the recommendation results of NLP models, and the attention mechanism in NLP models is a starting point. In NLP models, attention values ​​(or weights) can be used to represent the importance of each word, so attention values ​​can be considered as an explanation of the recommendation results. Although many works have visualized the attention values ​​in NLP models, they still cannot be directly used as an explicit explanation of the recommendation results, and there is no good way to use these attention values ​​as such an explanation. In fact, there are many possible ways to interpret the results of SR recommender systems. For example, using topics as input to a classification model or using attention values, but these methods may be unreliable.

[0119] Topic modeling is another good metric in Natural Language Processing (NLP) for extracting topics from text based on its inherent properties, but it's not currently commonly used in interpretable AI. However, topic models can sometimes be used for classification tasks, and the topics in a topic model can be an interpretation of a single category. Nevertheless, topic models are unreliable and can extract many meaningless words. Furthermore, since topic modeling is an unsupervised method, hyperparameters like the number of topics are important for the final performance. Intuitively, the human decision-making process based on text content is as follows: first, understand the characteristics of the text, and then use these properties to make a decision. Therefore, the attention value in an NLP model that collaborates with topic models can be considered as a model for this decision-making process.

[0120] Currently, reliably explaining artificial intelligence systems is not a trivial task, and much research has emerged on this topic in recent years. However, in text classification tasks, there is currently no reliable and comprehensive method for extracting explanatory information. To make recommender systems (e.g., SR recommender systems) explainable AI systems, it is first necessary to find ways to use attention values ​​from NLP models and to encode the topics of the text analyzed by topic models. Then, it is necessary to design ways to correctly model the decision-making process.

[0121] First, explanatory information extraction for NLP models (e.g., BERT models) has been investigated. For example, attention can be visualized to explain NLP models, but this explanation is unreliable. Intuitively, each head in an NLP model should represent an aspect of the model, but some heads in the model often lack concrete meaning and cannot be described, thus failing to serve as appropriate explanatory information. Therefore, a good approach to finding useful heads in NLP models is necessary. Next, topic models for explanatory information have been studied. Topic models can be used to complete classification tasks, and the topics analyzed by the topic model can serve as a form of explanation. However, this is not accurate enough and is not directly related to the prediction model. More specifically, if a user wants to obtain keywords related to the prediction results, these topics are inaccurate and difficult to interpret.

[0122] Next, the combined results of the NLP model and the topic model are considered. In most people's minds, classic AI methods that only output results without explanation are unreliable. Therefore, systems that output explanatory information along with predictions are important. When people make decisions based on text, they first try to understand the meaning of the text, and then use their knowledge and that understanding to make a decision. Therefore, in embodiments of this disclosure, the topic model can be used to understand the meaning of the text, while the NLP model (e.g., the BERT model) can be used to model the second step above. However, for the explanation part, a good method needs to be found to combine these two steps and provide reliable explanatory information.

[0123] Based on the above analysis and discussion, the solution disclosed herein can extract explanatory information from a trained NLP model (e.g., BERT model) using topic modeling. Specifically, a multi-label classification model can first be trained using an NLP model, and then a topic model (e.g., LDA model) can be trained. After the topic model is trained, new combinatorial embeddings (vectorization) are defined for the topics in the topic model, and then the weighted distance between the topics in the topic model and the words in the text, also known as the scaled distance, is calculated to identify the words in the input text that serve as explanatory information. See below for reference. Figure 17 This describes a recommendation system, such as the SR recommendation system, constructed according to embodiments of the present disclosure.

[0124] Figure 17 A schematic framework diagram of a recommendation system 1700 that simultaneously outputs recommended objects and explanatory information according to an embodiment of the present disclosure is shown. Figure 17 As shown, in some embodiments, the recommendation system 1700 can be... Figure 1 The recommendation system 1700 is implemented using the computing device 110, therefore it can be understood as... Figure 1 This is one example of the information processing system 100 depicted. However, in other embodiments, the various units or modules in the recommendation system 1700 may also be implemented in one or more other computing devices, or in remote computing devices, such as cloud computing devices.

[0125] exist Figure 17In the example, text 120 can be provided to computing device 110 as input to recommender system 1700. The NLP model (e.g., BERT model) 1710 and topic model 1720 of recommender system 1700 are trained models. Using these two models to process the input text 120, recommender system 1700 can provide object identifiers 1712 for the recommended objects, such as the ID of the recommended knowledge base. Furthermore, recommender system 1700 can also obtain attention values ​​from the NLP model (e.g., BERT model) for each word in the original text 120, multiple predefined topics 1722 (represented as a word distribution) from topic model 1720, and a topic distribution 1724 for the original text 120. These outputs from the two models, except for object identifiers 1712, can then be used by recommender system 1700 to generate explanatory information 1730. Explanatory information 1730 and object identifiers 1712 are the final output 1740 of recommender system 1700.

[0126] Specifically, in one aspect, the recommender system 1700 can utilize an NLP model 1710 to classify text 120 to determine the object associated with text 120 (e.g., a recommended knowledge base) among multiple predefined objects. In some embodiments, the NLP model 1710 can be a text classification model based on the BERT model. During the training of the BERT model, all layers can be unfrozen to fine-tune the model using the specific task of the recommender system 1700, which associates attention values ​​with the final result (classification label). In the recommender system 1700, the BERT model is used in a multi-label classification task.

[0127] In other embodiments, the NLP model 1710 may also be other known or future-developed machine learning-based natural language processing models. Based on its understanding of the natural language in the text 120, the NLP model 1710 can generate object identifiers 1712 and attention weights 1714. Object identifiers 1712 indicate the recommended objects that the recommendation system 1700 makes based on the input text 120, while attention weights 1714 indicate the set of words with higher attention weights in determining the recommended objects (e.g., ...). Figure 1 The set of keywords described in the text is 135.

[0128] In some cases, the weights corresponding to a word in the attention weights 1714 of the NLP model 1710 might be determined based on a self-attention mechanism, such as a weight vector comprising multiple components. However, the user of the recommender system 1700 might want a more explicit explanation, such as keywords in a sentence in text 120. This means that the recommender system 1700 can make a decision because it "sees" those keywords. In the original attention matrix of the attention weights 1714 of the NLP model 1710, the weight information is related to the attention corresponding to each word, while the recommender system 1700 needs to find the attention corresponding to the final output recommendation. Therefore, the recommender system 1700 can calculate the average of these attentions to obtain the final attention value. Then, the recommender system 1700 can retain the words with the first few attention values, which are the original keywords (e.g., ...). Figure 1 The set of keywords described in the text is 135.

[0129] On the other hand, the recommender system 1700 can utilize topic model 1720 to analyze the topics involved in text 120. In some embodiments, topic model 1720 can be a Latent Dirichlet Allocation (LDA) model. Regarding the training example of topic model 1720, during the training of the aforementioned NLP model (e.g., BERT model), the original dataset may have more than 3000 labels, but some labels have similar topics. Therefore, 2000 can be selected as the number of topics for topic model 1720, and an LDA model can be selected as the topic model 1720. In other embodiments, topic model 1720 can also be other known or future-developed topic models. Based on the statistics and analysis of the topics involved in text 120, topic model 1720 can generate multiple predetermined topics 1722 and corresponding topic distributions 1724 involved in text 120.

[0130] Then, the recommender system 1700 can determine parsed information 1730 based on attention weights 1714, multiple predefined topics 1722, and corresponding topic distributions 1724. For example, parsed information 1730 may include a target word set 137 in the text 120. The target word set 137 includes words that are given high attention weights when determining the recommended object (e.g., the recommended knowledge base) and are highly relevant to the topic of the text 120. Specifically, the topic distribution 1724 of the text 120 may be a probability distribution of the predefined topics 1722, and each predefined topic is a word distribution. Therefore, the recommender system 1700 can utilize this word distribution to compute predefined topic embeddings. To obtain embeddings for predefined topics, the recommender system 1700 can first obtain word embeddings of the words for the predefined topics (e.g., using a BERT model). Then, the recommender system 1700 can obtain a weighted sum of these word vectors as the topic vector in subsequent weighted distance calculations.

[0131] Next, the recommender system 1700 can calculate the weighted distances between words in the text 120 and multiple predefined topics 1722. As noted above, if only an NLP model (e.g., a BERT model) is used to determine keywords, many meaningless keywords (e.g., special tags generated during BERT inference) will be included in the keyword set. Therefore, in embodiments of this disclosure, the recommender system 1700 can filter out these meaningless words using topic vectors and topic distributions of the predefined topics.

[0132] In other words, the weighted distance between words in the text and topic vectors can be used as a filter in the embodiments of this disclosure. Specifically, firstly, several words with the highest average weight are converted into word embeddings, and then the weighted distance for a certain word (word1) is: distance1 = p1 × dis(w1, topic1) + ... + p n ×dis(w1,topic n The distance function `dis()` can use cosine similarity, where `p` represents the probability of a predetermined topic, `topic` represents the topic vector of the predetermined topic, and `w` represents the word vector. This weighted distance can refer to the distance between attention meaning and topic meaning. If a word belongs to a topic and has a high attention value, then that word should be used as the final keyword for explaining information 1730, that is, the keywords with the top k lowest distances (where k is the number of words to be retained) are kept.

[0133] Then, the recommender system 1700 can provide both the object identifier 1712 and the explanation information 1730 as outputs to the user of the recommender system 1700. Therefore, the user of the recommender system 1700 not only knows the object recommended by the recommender system 1700 (e.g., a knowledge base), but also knows the reason why the object is recommended, thus having more confidence in the correctness of the recommended object, and consequently obtaining a better user experience and higher efficiency.

[0134] In summary, while a good recommendation system can guide junior engineers in traditional customer service systems without requiring the time of senior or junior engineers, interpreting the recommendation results reasonably is not straightforward. However, conventional methods for explaining the decision-making process are either inadequate or ignore the original recommendation model. Therefore, embodiments of this disclosure propose a novel approach to explain classification models (e.g., BERT models) using topic models. Specifically, attention values ​​in the machine learning model are used in an innovative averaging manner, and a novel topic embedding method is introduced to represent the topics of the topic model. Based on these innovations, weighted distance is used to filter keywords generated solely using attention values, thereby obtaining more meaningful keywords.

[0135] Therefore, the novel method disclosed herein considers both the original model used in the recommender system (BERT) and the reliability of the explanation (topic model). Specifically, attention values ​​are used to provide initial explanations of why the original model gives this result, and then the topic model is used to make this explanation more reasonable. In the example experiments, the explanation becomes more reliable after some meaningless words are eliminated by using the topic model.

[0136] The technical advantages and contributions of the embodiments of this disclosure lie in firstly providing a comprehensive model that outputs not only recommendation results but also explanatory information. For example, in an SR recommendation system, explanations are provided simultaneously when recommending knowledge bases to engineers. Specifically, the reasons for recommending these knowledge bases can be given by showing important words in the service request. Furthermore, novel algorithms that modify the way attention values ​​are used are provided. For example, an average attention value is used to represent the importance of each word. In addition, novel algorithms for generating topic embeddings are provided. For example, word embeddings and topic distributions are used to obtain topic embeddings (topic vectors), thereby encoding topic information in a better way. Moreover, novel combinations or novel algorithms of topic models and NLP models (e.g., the BERT model) are provided. This helps BERT filter out initially confusing explanations. For example, topics generated by the topic model are used to filter explanation words generated using attention values. Specifically, the weighted distance between topic vectors and word vectors generated using attention can be used to accomplish this filtering.

[0137] Figure 18 A comparison is shown between the target word set output by the information processing method according to embodiments of the present disclosure and the keywords output by other keyword determination schemes. Figure 18 In the example experiment, the input text 1810 is the English sentence "Post the upgrade customer is facing a single path host issue". When keywords are determined solely based on attention weights, the four keywords 1820 with the highest attention weights are "[SEP]", "[CLS]", "host", and "issue". It is evident that keyword 1820 fails to provide a good interpretation of the output, as it even contains special markers used by the NLP model for intermediate processing (such as "[CLS]") and other irrelevant words.

[0138] When using only topic modeling to determine keywords, the four keywords 1830 most relevant to the topic of input text 1810 are "problem," "root," "please," and "update." It is evident that keyword 1830 also fails to provide a good explanation of the output, as directly using topic-related words to explain why the target object was recommended is not clear enough, and it also includes words not included in the input text 1810. The target word set determined using embodiments of this disclosure includes the words "issue," "single," "path," and "host," and this target word set is clearer and easier to understand than the results of other keyword determination schemes. Therefore, the results of this example experiment clearly demonstrate the advantages of the combined scheme of this disclosure in explaining the target object.

[0139] Figure 19 A block diagram of an example device 1900 that can be used to implement embodiments of the present disclosure is shown. In some embodiments, device 1900 may be an electronic device that can be used to implement... Figure 1 The computing device 110 in the middle. For example... Figure 19 As shown, device 1900 includes a central processing unit (CPU) 1901, which can perform various appropriate actions and processes according to computer program instructions stored in read-only storage device (ROM) 1902 or loaded from storage unit 1908 into random access storage device (RAM) 1903. Various programs and data required for the operation of device 1900 can also be stored in RAM 1903. CPU 1901, ROM 1902, and RAM 1903 are interconnected via bus 1904. Input / output (I / O) interface 1905 is also connected to bus 1904.

[0140] Multiple components in device 1900 are connected to I / O interface 1905, including: input unit 1906, such as keyboard, mouse, etc.; output unit 1907, such as various types of monitors, speakers, etc.; storage unit 1908, such as disk, optical disk, etc.; and communication unit 1909, such as network card, modem, wireless transceiver, etc. Communication unit 1909 allows device 1900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0141] The various processes and procedures described above, such as example methods or example procedures, may be executed by processing unit 1901. For example, in some embodiments, the various example methods or example procedures may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1900 via ROM 1902 and / or communication unit 1909. When the computer program is loaded into RAM 1903 and executed by CPU 1901, one or more steps of the example methods or example procedures described above may be performed.

[0142] As used herein, the term "comprising" and similar expressions should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. This document may also include other explicit and implicit definitions.

[0143] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculation, computation, processing, derivation, investigation, searching (e.g., looking in a table, database, or other data structure), ascertainment, etc. Furthermore, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Additionally, "determine" can include parsing, selecting, choosing, building, etc.

[0144] It should be noted that the embodiments of this disclosure can be implemented using hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, code provided on programmable memory or a data carrier such as an optical or electronic signal carrier.

[0145] Furthermore, although the operation of the methods of this disclosure is described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Rather, the steps depicted in the flowcharts may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps. It should also be noted that the features and functions of two or more devices according to this disclosure may be embodied in one device. Conversely, the features and functions of one device described above may be further divided and embodied by multiple devices.

[0146] While this disclosure has been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the specific embodiments disclosed. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. An information processing method, comprising: Receive text from users of the storage system, including service requests; A first machine learning model of a machine learning system implemented using a processor, and based on multiple weights corresponding to multiple words in the text, determines a target object associated with the text and a set of keywords in the multiple words related to determining the target object from a predetermined set of objects; The second machine learning model of the machine learning system implemented by the processor, and the set of target words related to the text topic of the text in the set of keywords; The machine learning system implemented by the processor outputs the target word set in association with the identifier of the target object, wherein outputting the target word set and the identifier of the target object includes: generating at least one visualization in which the target word set is associated with the identifier of the target object in a visually perceptible manner; The output target word set and the identifier of the target object are processed to generate corresponding recommendations; The recommendation is sent to the user as at least part of a service response configured to resolve the service request in the text; and Based on the recommendation, one or more actions related to the storage system are performed to resolve the service request in the text; The first machine learning model of the machine learning system implemented by the processor includes a trained text classification model for natural language processing; The second machine learning model in the machine learning system implemented by the processor includes a topic model; The method of using the second machine learning model to determine the target word set includes: Determine the relevance of a first number of words in the keyword set to a first number of text topics; Among the first number of text topic relevances, a second number of text topic relevances are determined in descending order of magnitude; and From the first number of words, determine a second number of words that correspond to the second number of text topic relevances; The plurality of weights includes a first weight of a first word among the plurality of words, the first weight comprising a plurality of first weight components, the plurality of first weight components being determined based on the plurality of word relevances between the plurality of words and the first word; and Generating the at least one visualization further includes generating a visual representation of the corresponding word relevance among the plurality of word relevances.

2. The method according to claim 1, wherein determining the keyword set comprises: Determine multiple average weights corresponding to the multiple weights, each average weight being determined based on multiple weight components of the corresponding weight; Among the plurality of weighted averages, a first number of weighted averages are determined in order of magnitude; as well as From the plurality of words, determine a first number of words that correspond to the average weight of the first number.

3. The method of claim 1, wherein determining the first number of text topic relevances comprises: The text topics are determined to be multiple probabilities of multiple predetermined topics; Determine the relevance of the first word among the first number of words to the multiple predetermined topics; as well as Based on the multiple predetermined topic relevances and the multiple probabilities, the first text topic relevance corresponding to the first word is determined.

4. The method of claim 3, wherein determining the relevance of the plurality of predetermined topics comprises: Determine the first set of words and the corresponding first probability set included in the first predetermined topic among the plurality of predetermined topics; Determine the first word vector set of the first word set; Based on the first probability set and the first word vector set, determine the first predetermined topic vector of the first predetermined topic; as well as Based on the word vector of the first word and the first predetermined topic vector, the relevance of the first predetermined topic corresponding to the first predetermined topic is determined.

5. The method of claim 3, wherein determining the plurality of probabilities comprises: The plurality of probabilities are determined based on the topic model used to determine the text topic, wherein the topic model is trained using a set of texts associated with the plurality of predetermined objects.

6. The method according to claim 1, wherein outputting the target word set comprises: The target word set is output as explanatory information about the association between the text and the target object.

7. The method of claim 1, wherein the text is descriptive text about a fault in the storage system, and the plurality of predetermined objects are predetermined knowledge bases, the plurality of knowledge bases including different sets of text associated with resolving different types of faults in the storage system.

8. An electronic device, comprising: At least one processor; as well as At least one memory storing computer program instructions, said at least one memory and said computer program instructions being configured, together with said at least one processor, to cause the electronic device to: Receive text from users of the storage system, including service requests; A first machine learning model of a machine learning system implemented using a processor, and based on multiple weights corresponding to multiple words in the text, determines a target object associated with the text and a set of keywords in the multiple words related to determining the target object from a predetermined set of objects; The second machine learning model of the machine learning system implemented by the processor, and the set of target words related to the text topic of the text in the set of keywords; The machine learning system implemented by the processor outputs the target word set in association with the identifier of the target object, wherein outputting the target word set and the identifier of the target object includes: generating at least one visualization in which the target word set is associated with the identifier of the target object in a visually perceptible manner; The output target word set and the identifier of the target object are processed to generate corresponding recommendations; The recommendation is sent to the user as at least part of a service response configured to resolve the service request in the text; and Based on the recommendation, one or more actions related to the storage system are performed to resolve the service request in the text; The first machine learning model of the machine learning system implemented by the processor includes a trained text classification model for natural language processing; The second machine learning model in the machine learning system implemented by the processor includes a topic model; Determining the target word set includes: Determine the relevance of a first number of words in the keyword set to a first number of text topics; Among the first number of text topic relevances, a second number of text topic relevances are determined in descending order of magnitude; and From the first number of words, determine a second number of words that correspond to the second number of text topic relevances; The plurality of weights includes a first weight of a first word among the plurality of words, the first weight comprising a plurality of first weight components, the plurality of first weight components being determined based on the plurality of word relevances between the plurality of words and the first word; and Generating the at least one visualization further includes generating a visual representation of the corresponding word relevance among the plurality of word relevances.

9. The electronic device of claim 8, wherein the at least one memory and the computer program instructions are configured, together with the at least one processor, to cause the electronic device to determine the set of keywords by: Determine multiple average weights corresponding to the multiple weights, each average weight being determined based on multiple weight components of the corresponding weight; Among the plurality of weighted averages, a first number of weighted averages are determined in order of magnitude; as well as From the plurality of words, determine a first number of words that correspond to the average weight of the first number.

10. The electronic device of claim 8, wherein the at least one memory and the computer program instructions are configured, together with the at least one processor, to cause the electronic device to determine the first number of text topic relevances by: The text topics are determined to be multiple probabilities of multiple predetermined topics; Determine the relevance of the first word among the first number of words to the multiple predetermined topics; as well as Based on the multiple predetermined topic relevances and the multiple probabilities, the first text topic relevance corresponding to the first word is determined.

11. The electronic device of claim 10, wherein the at least one memory and the computer program instructions are configured, together with the at least one processor, to cause the electronic device to determine the relevance of the plurality of predetermined topics by: Determine the first set of words and the corresponding first probability set included in the first predetermined topic among the plurality of predetermined topics; Determine the first word vector set of the first word set; Based on the first probability set and the first word vector set, a first predetermined topic vector for the first predetermined topic is determined; and Based on the word vector of the first word and the first predetermined topic vector, the relevance of the first predetermined topic corresponding to the first predetermined topic is determined.

12. The electronic device of claim 10, wherein the at least one memory and the computer program instructions are configured, together with the at least one processor, to cause the electronic device to determine the plurality of probabilities by: The plurality of probabilities are determined based on the topic model used to determine the text topic, wherein the topic model is trained using a set of texts associated with the plurality of predetermined objects.

13. The electronic device of claim 8, wherein the at least one memory and the computer program instructions are configured, together with the at least one processor, to cause the electronic device to output the target word set by: The target word set is output as explanatory information about the association between the text and the target object.

14. The electronic device of claim 8, wherein the text is descriptive text about a fault in the storage system, and the plurality of predetermined objects are predetermined plurality of knowledge bases, the plurality of knowledge bases including different sets of text associated with resolving different types of faults in the storage system.

15. A computer program product tangibly stored on a non-volatile computer-readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform the steps of the method according to any one of claims 1-7.

Images