Method, device, apparatus, and medium for updating model parameters

Abstract

Embodiments of the present disclosure relate to methods, apparatuses, devices, and computer-readable storage media for updating model parameters. The method includes: determining a set of concept pairs for training a concept analysis model based on the correlation between concepts in a predefined concept system, each concept pair in the set of concept pairs including a first concept and a second concept associated with the first concept; and applying the set of concept pairs to the concept analysis model so that the concept analysis model updates the parameter set of the concept analysis model based at least on the set of concept pairs, thereby improving the measure of matching of the first concept of each concept pair in the set of concept pairs with the second concept through the concept analysis model. Thus, in the text processing task, similar information on the concept hierarchy can be introduced for each concept word, thereby enriching the feature representation of the concept word, and further improving the effect of the text processing algorithm.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of text processing, and more particularly to methods, apparatuses, devices, and media for updating model parameters. Background Art

[0002] In text processing tasks, text features must possess good generalization performance. This means that two similar but different texts should have similar semantic features. This allows for the incorporation of semantic information into text processing tasks, thereby improving the effectiveness of text processing algorithms. Concepts are an important component of text features, so generalization can be achieved by representing the conceptual relationships within text in an efficient and computationally efficient manner.

[0003] Traditional concept relationship feature representation usually adopts two solutions. The first solution is the context-based distributed word vector feature. This solution uses the context in which the word appears in the text corpus as the training corpus, and finally trains the vector representation of each word. The second solution is based on the representation learning method of the knowledge graph. This solution trains the vector representation of the entity in the knowledge graph based on the triple set in the existing knowledge graph. However, since the first solution trains word vectors based on the context corpus and calculates the similarity of words through the contextual co-occurrence information of words in the corpus, it cannot capture the hierarchical generalization relationship of concepts. Since the second solution trains entity vectors based on the knowledge graph and calculates the similarity of entities through the triple co-occurrence information in the knowledge graph, it also cannot capture the hierarchical generalization relationship of concepts. Summary of the Invention

[0004] According to an exemplary embodiment of the present disclosure, a solution is provided for embedding a tree-like hierarchical structure of a concept system into a vector space to obtain a text feature representation of concept generalization in the form of a feature vector set.

[0005] In a first aspect of the present disclosure, a method for updating model parameters is provided. The method includes determining a set of concept pairs for training a concept analysis model based on the association between concepts in a predefined concept system, each concept pair in the set of concept pairs including a first concept and a second concept associated with the first concept; and applying the set of concept pairs to the concept analysis model so that the concept analysis model updates a parameter set of the concept analysis model based on at least the set of concept pairs, thereby improving a measure of matching of the first concept of each concept pair in the set of concept pairs with the second concept through the concept analysis model.

[0006] In a second aspect of the present disclosure, a device for updating model parameters is provided. The device includes: a concept pair set determination module, configured to determine a set of concept pairs for training a concept analysis model based on the association between concepts in a predefined concept system, each concept pair in the set of concept pairs including a first concept and a second concept associated with the first concept; and a concept pair set application module, configured to apply the set of concept pairs to the concept analysis model, so that the concept analysis model updates a parameter set of the concept analysis model based on at least the set of concept pairs, thereby improving the measure of matching of the first concept of each concept pair in the set of concept pairs with the second concept through the concept analysis model.

[0007] In a third aspect of the present disclosure, an electronic device is provided. The electronic device includes: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, causes the one or more processors to implement the method according to the first aspect of the present disclosure.

[0008] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.

[0009] It should be understood that the contents described in the Summary of the Invention section are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0011] Figure 1 A schematic diagram illustrating an example environment 100 in which embodiments of the present disclosure may be implemented is shown;

[0012] Figure 2 A first block diagram 200 illustrating an example conceptual system according to some embodiments of the present disclosure is shown;

[0013] Figure 3 A second block diagram 300 illustrating an example conceptual architecture according to some embodiments of the present disclosure is shown;

[0014] Figure 4 A structural diagram 400 of an example character concept system according to some embodiments of the present disclosure is shown;

[0015] Figure 5 1 shows a flow chart of a method 500 for updating model parameters according to some embodiments of the present disclosure;

[0016] Figure 6 1 shows a flowchart of a method 600 for determining concept pairs according to some embodiments of the present disclosure;

[0017] Figure 7 A schematic block diagram of an apparatus 700 for updating model parameters according to some embodiments of the present disclosure is shown; and

[0018] Figure 8 A block diagram of a computing device 800 is shown in which embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0019] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0020] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, i.e., "including but not limited to." The term "based on" should be understood as "based at least in part on." The term "some embodiments" or "the embodiment" should be understood as "at least some embodiments." The terms "first," "second," etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0021] Traditional concept relationship feature representation cannot capture the hierarchical generalization relationship of concepts, and therefore cannot fully reflect the hierarchical and logical relationships between concepts, making it impossible to meet the needs of text processing tasks such as text classification and semantic parsing that require the introduction of concept hierarchical and logical features.

[0022] In response to the above problems, the embodiments of the present disclosure utilize the hierarchical structure of concepts in the concept system to represent the semantic structure of concepts, and embed the hierarchical and logical structures into the vector space used to represent the concept relationship features, so that the set of generated feature vectors can capture the hierarchical similarity between concepts, thereby introducing the hierarchical and logical features of concepts in the text processing task, thereby improving the generalization ability of the text processing algorithm.

[0023] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0024] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure may be implemented is shown. This example environment 100 primarily includes a model parameter updating device 110. It should be understood that the structure and functionality of the environment 100 are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure. Embodiments of the present disclosure may also be applied to environments having different structures and / or functions.

[0025] like Figure 1 As shown in , first, the concept system 101 is provided to the model parameter updating device 110.

[0026] In some embodiments, the concept system 101 may be a tree-like concept hierarchical network with a directed acyclic graph structure constructed in advance. Specifically, the structure of the concept system 101 may be as follows: Figures 2 to 4 The first structural diagram 200 of an exemplary concept system, the second structural diagram 300 of an exemplary concept system, and the structural diagram 400 of an exemplary person concept system according to some embodiments of the present disclosure are shown. It should be understood that the structure and form of the concept system 101 in the present disclosure is not limited to a tree-like concept hierarchical network with a directed acyclic graph structure, but can have any appropriate organizational form.

[0027] like Figure 2 As shown, the first structural diagram 200 of the example concept system includes a plurality of nodes A 201, B 202, C 203, D 204, E 205, F 206, G 207, H 208, I 209, and J 210. Each of the nodes A 201 to J 210 represents a concept in the concept system, and the directed edges with arrows between the nodes can represent the association between the concepts, such as a hierarchical relationship, a logical relationship, and / or any other appropriate relationship. An example of a hierarchical relationship can include, for example, if there are three semantic concepts: character, actor, and professional actor, the concept "character" has the largest coverage and can include the concept "actor", and the concept "actor" has a smaller coverage than the concept "character" but can include the concept "professional actor". At this point, it can be considered that the above and below concepts have a hierarchical relationship, where the concept "character" is the most superordinate, and is the superordinate concept of the concepts "actor" and "professional actor". The concept "actor" is the second most superordinate, and is the subordinate concept of the concept "character" and the superordinate concept of the concept "professional actor". The concept "professional actor" is the most subordinate, and is the subordinate concept of the concepts "character" and "actor". It should be understood that the above-mentioned hierarchical relationship can also be used in geographical scope and other fields. Logical relationships can include, for example, dependency relationships and causal relationships.

[0028] exist Figure 2In the illustrated embodiment, when the directed edges represent a hierarchical relationship between concepts, the edges pointing from nodes E 205 and F 206 to node D 204 indicate that the concepts represented by nodes E 205 and F 206 are subordinate concepts of the concept represented by node D 204, and the edge pointing from node D 204 to node B 202 indicates that the concept represented by node D 204 is a subordinate concept of the concept represented by node B 202, and further, the concepts represented by nodes E 205 and F 206 are also subordinate concepts of the concept represented by node B 202. Nodes C 203, D 204, and E 205 are not directly or indirectly connected by one or more edges in a single direction, and therefore, no hierarchical relationship exists between nodes C 203, D 204, and E 205. It should be understood that the structure of the first structural diagram 200 of the exemplary concept system is described for exemplary purposes only and does not imply any limitation on the scope of the present disclosure. The first structural diagram 200 of the exemplary conceptual system may also include more or fewer nodes, and the nodes may have other hierarchical relationships represented by directed graphs.

[0029] According to an embodiment of the present disclosure, the concept system 101 (which can be embodied as, for example, Figure 2 ). In this case, the concept node itself can be considered as a symbol, that is, a symbolic representation, in which each concept node can be represented by a feature vector, and such a vector representation can be understood as a parameter set for the concept node. The aforementioned vector space can be a vector space of 200 dimensions to 500 dimensions. In this case, the feature vector of each concept can be represented as an array of floating-point numbers whose number is equal to the dimension of the vector space. For example, when the vector space is a 200-dimensional vector space, each concept can be represented as an array of 200 floating-point numbers to represent the feature vector for this concept. Specifically, the array of floating-point numbers can represent the position of this concept node in the vector space.

[0030] like Figure 2 As shown, node G 207 has two upper nodes B 202 and H 208, that is, the concept represented by node G 207 has a direct semantic similarity relationship with the concepts represented by nodes B 202 and H 208, and the concepts represented by nodes B 202 and H 208 are all upper concepts of the concept represented by node G 207. According to the embodiment of the present disclosure, in the actual calculation process, the semantic similarity relationship between node G 207 and the subtree including node B 202H and the subtree including node H 208 can be captured at the same time. Therefore, Figure 2 The first block diagram 200 of the example conceptual system shown in FIG is transformed into Figure 3The second structural diagram 300 of the exemplary conceptual system shown in FIG. 3 , that is, the structure in the second structural diagram 300 may be an alternative structure to the structure in the first structural diagram 200 .

[0031] like Figure 3 As shown, the second structural diagram 200 of the exemplary conceptual system includes a plurality of nodes A' 301 , B' 302 , C' 303 , D' 304 , E' 305 , F' 306 , G1 307 - 1 , G2 307 - 2 , H' 308 , I' 309 , and J' 310 . Figure 3 and Figure 2 The main difference is Figure 2 Node G 207 shown in Figure 3 302H and the subtree including node B'302H and the subtree including node H'308. At this time, by calculating the semantic similarity between node G1 307-1 and the nodes of the subtree including node B'302H, and between node G2 307-2 and the nodes of the subtree including node H'308, the feature vectors of nodes G1 307-1 and G2 307-2 can be obtained respectively. The feature vectors of nodes G1 307-1 and G2 307-2 can be used as the corresponding feature vectors. Figure 2 The feature vector of the node G 207 shown in FIG can also be calculated by performing a weighted average calculation on the feature vectors of the nodes G1 307-1 and G2 307-2, thereby representing the feature vector corresponding to the node G1 307-1 and G2 307-2 as a whole. Figure 2 The semantic similarity of the feature vector of node G 207 in the subtree including node B' 302H and the subtree including node H' 308 is shown in FIG.

[0032] Figure 4 A structural diagram 400 of an exemplary character concept system according to some embodiments of the present disclosure is shown. Specifically, Figure 4 The structure diagram 400 of the example person concept system shown in FIG corresponds to Figure 2 The first structural diagram 200 of the exemplary conceptual system shown in FIG. 1 is a specific example of the first structural diagram 200 of the exemplary conceptual system. Figure 4 In Figure 1 Each concept node in the first structural diagram 200 of the exemplary concept system shown in FIG has added a specific Chinese concept. It should be understood that the added concepts are not limited to Chinese, but can be in any language. Figure 4 As shown, the structural diagram 400 of the example character concept system includes multiple node characters 401, actors 402, talented actors 403, professional actors 404, voice actors 405, stuntmen 406, movie stars 407, stars 408, sports stars 409 and football stars 410. Figure 4The hierarchical relationship between each concept node in Figure 1 The hierarchical relationships between the various concept nodes shown in are the same and will not be repeated here.

[0033] According to an embodiment of the present disclosure, each node in the conceptual system 101 provided to the model parameter updating device 110 may have a unique serial number, such as 0, 1, 2, 3, ..., so that the corresponding node can be uniquely identified by the serial number. It should be understood that the conceptual system 101 may not itself assign serial numbers to the nodes therein, but rather, after being received by the model parameter updating device 110, the model parameter updating device 110 assigns a unique serial number to each node in the conceptual system 101.

[0034] like Figure 1 As shown, after receiving the concept system 101, the model parameter updating device 110 can input model input information 102 to the concept analysis model 120 (concept2vec training model). The model input information 102 includes model initialization information 103, training samples 104, and training end conditions 105, and at least a portion of the model input information 102 can be determined by the model parameter updating device 110 at least in part based on the received concept system 101.

[0035] After receiving the model input information 102 from the model parameter updating device 110, the concept analysis model 120 can initialize the concept analysis model 120 based on the model initialization information 103, train the concept analysis model 120 based on the training sample 104 to continuously update the parameter set in the concept analysis model 120, and end the training of the concept analysis model 120 when the training end condition 105 is met. At this time, the updated parameter set is the set of feature vectors of each concept node in the concept system 101 that meets the user requirements.

[0036] The following will be further described in detail with reference to the accompanying drawings. Figure 1 The working principles of the model parameter updating device 110 and the concept analysis model 120 are shown.

[0037] Figure 5 FIG. 5 is a flow chart showing a method 500 for updating model parameters according to some embodiments of the present disclosure. For example, the method 500 may be performed as follows: Figure 1 The model parameter updating device 110 shown in FIG. Figures 1 to 4 Detailed description of each action of method 500 is provided. It should be understood that method 500 may also include additional actions not shown and / or may omit actions shown. The scope of the present disclosure is not limited in this respect.

[0038] In block 510, the model parameter updating device 110 determines a set of concept pairs (eg, Figure 1 The training sample 104 shown). Concept pair This concept will be referred to below Figure 6 and Figure 1 In some embodiments, the concept analysis model 120 may be a classification model and may use a machine learning model such as SkipGram, convolutional neural network, or CBOW.

[0039] Figure 6 FIG. 6 is a flow chart showing a method 600 for determining concept pairs according to some embodiments of the present disclosure. For example, the method 600 may be viewed as follows: Figure 5 An example implementation of block 510 in method 500 is shown. It should be understood that method 600 may also include additional actions not shown and / or may omit actions shown. The scope of the present disclosure is not limited in this respect.

[0040] like Figure 6 As shown, in box 610, the model parameter updating device 110 determines a subset of concepts corresponding to each concept in the concept system 101 from the concept system 101. According to an embodiment of the present disclosure, the subset of concepts corresponding to each concept can be a set of concepts that have relevance to this concept. The following describes the determination step in box 610 by taking the semantic similarity of the relationship between the relevance as an example. It should be understood that the relevance can also include the logical relationship between concepts. In this case, the more direct or stronger the logical relationship between concepts, the higher the relevance between the concepts.

[0041] As Figures 2 to 4 As shown in , the concept system 101 in the present disclosure can be a tree-structured concept system. The semantic similarity of nodes in the tree-structured concept system is represented as follows: first, the most similar node set of each node (node) is defined as all its descendant nodes (son(node)). In this way, if there is a hierarchical relationship between two nodes, the most similar node set of the lower node is a subset of the most similar node set of the upper node, and there is a transitive hierarchical generalization relationship between the two; secondly, according to the most similar node set of each node and the hierarchical relationship of the nodes, a partial order relationship of the similarity between each node and other nodes in the tree is constructed. This partial order relationship includes, for example: {descendant node set}>{parent node}>{other descendant node sets of the parent node}>{grandfather node}>{other descendant node sets of the grandfather node}>…, wherein the similarity (i.e., correlation) between the nodes in the set on the left side of ">" and the nodes for which these sets are constructed is higher than the similarity between the nodes in the set on the right side of ">" and the nodes for which these sets are constructed.

[0042] For example, in Figure 3 In the second structural diagram 300 of the exemplary concept system, the similarity partial order between node D'304 and other nodes is: {E'305, F'306}>{B'302}>{C'303, G1 307-1}>{A'301}>{H'308, I'309, G2307-2, J'310}. Figure 4 The structure diagram 400 of the example character concept system shown, if Figure 3 The concept corresponding to the node D'304 is Figure 4 Professional actor 404, then Figure 4 The partial order of similarity between the concept node "professional actor 404" and other concept nodes in the structure diagram 400 is: {voice actor 405, stuntman 406}>{actor 402}>{capable actor 403, movie star 407}>{character 401}>{star 408, sports star 409, ball star 410}; and the partial order of similarity between the concept node "star 408" and other concept nodes in the structure diagram 400 is: {sports star 409, ball star 410, movie star 407}>{character 401}>{actor 402, professional actor 404, voice actor 405, stuntman 406, capable actor 403}.

[0043] In the above manner, based on the partial order relationship of the similarity between each node and other nodes in the tree structure defined above, for each concept node (nodeC), its training sample set (train(nodeC)) can be constructed, that is, the subset of concepts corresponding to each concept in the concept system 101 can be determined.

[0044] In block 620, the model parameter updating device 110 pairs each concept in the concept system 101 with a concept in the corresponding subset of concepts as a concept pair in a set of concept pairs. Figure 4Taking the similarity partial order of the concept node "professional actor 404" in the structure diagram 400 and other concept nodes as follows: {voice actor 405, stuntman 406}>{actor 402}>{capable actor 403, movie star 407}>{character 401}>{star 408, sports star 409, ball star 410} as an example, the concept pairs in the set of concept pairs that can be constructed for the concept node "professional actor 404" include (professional actor 404, voice actor 405), (professional actor 404, stuntman 406), (professional actor 404, actor 402), (professional actor 404, captivating actor 403), (professional actor 404, movie star 407), (professional actor 404, character 401), (professional actor 404, star 408), (professional actor 404, sports star 409) and (professional actor 404, ball star 410). It can be seen that the above-mentioned concept pairs include the first concept "professional actor 404" and the second concept associated with the first concept "voice actor 405", "stuntman 406", "actor 402", "powerful actor 403", "movie star 407", "character 401", "star 408", "sports star 409" or "ball star 410".

[0045] According to an embodiment of the present disclosure, since the concepts in the concept system 101 may reach the order of millions, if all possible constructed concept pairs are used to train the concept analysis model 120, it may result in an excessive amount of computation. At the same time, if too few concept pairs are used to train the concept analysis model 120, it may result in insufficient training of the concept analysis model 120. Therefore, according to an embodiment of the present disclosure, the minimum sample size (minTrain) required for each node can be defined. For each concept node (nodeC), all its descendant node sets (son(nodeC)) are obtained. If the number of nodes in the descendant node set (son(nodeC)) is greater than or equal to the minimum sample size (minTrain), the training sample set (train(nodeC)) is made equal to the descendant node set (son(nodeC)), because the descendant nodes are the nodes with the highest correlation. It should be understood that at this time, only the nodes with a number equal to the minimum sample size (minTrain) can be selected from the descendant node set (son(nodeC)) in a random or sequential manner as the training sample set (train(nodeC)). If the number of nodes in the descendant node set (son(nodeC)) is less than the minimum sample size (minTrain), then the training samples are supplemented according to the similarity partial order relationship defined above until the number of nodes in the training sample set (train(nodeC)) is greater than or equal to the minimum sample size (minTrain).

[0046] For example, still Figure 4 The similarity partial order between the concept node "professional actor 404" and other concept nodes in the structure diagram 400 is: {voice actor 405, stunt actor 406}>{actor 402}>{capable actor 403, movie star 407}>{character 401}>{star 408, sports star 409, ball star 410}. For example, assuming that the minimum sample size (minTrain) = 3, the training sample set (train(nodeC = "professional actor 404")) = {voice actor 405, stunt actor 406, actor 402}; assuming that the minimum sample size (minTrain) = 4, the training sample set (train(nodeC = "professional actor 404")) = {voice actor 405, stunt actor 406, actor 402, capable actor 403, movie star 407}. It should be understood that the concepts in the constructed training sample set (train(nodeC="professional actor 404")) have a high correlation with the concept "professional actor 404". At this time, three or four concept pairs can be constructed for the concept node "professional actor 404".

[0047] return Figure 5 , the method 500 proceeds to box 520, and the model parameter updating device 110 applies the set of concept pairs determined in box 510 to the concept analysis model 120, so that the concept analysis model 120 updates the parameter set of the concept analysis model 120 based on at least the set of concept pairs.

[0048] According to an embodiment of the present disclosure, the parameter set may include a set of feature vectors. Therefore, before applying the set of concept pairs determined in block 510 to the concept analysis model 120, the model parameter updating device 110 may provide the concept analysis model 120 with the number and dimension of the feature vectors to be generated (e.g., Figure 1 The model initialization information 103 shown in FIG. 1 is provided so that the concept analysis model 120 can generate a set of feature vectors based on the number and dimension and then use them for training. It should be understood that the concept analysis model 120 can also be pre-set to be suitable for feature vectors of a predetermined number and dimension, so that a set of feature vectors can be automatically generated without the model parameter updating device 110 providing the number and dimension of the feature vectors to be generated.

[0049] According to some embodiments of the present disclosure, the concept analysis model 120 may generate a set of feature vectors in the form of a matrix, where the number of rows (X) of the matrix represents the number of feature vectors that need to be updated, each feature vector corresponds to a concept, and the number of columns (Y) of the matrix represents the dimension of each feature vector, that is, the dimensional vector space in which these feature vectors reside. The concept analysis model 120 may generate this matrix, i.e., a set of feature vectors, in the form of, for example, a random setting.

[0050] As previously described, each concept pair in the set of concept pairs determined in box 510 includes a first concept and a second concept, and each concept (i.e., a node) in the concept system 101 has a unique serial number, so the model parameter updating device 110 applies the set of concept pairs determined in box 510 to the concept analysis model 120, which may include providing the serial numbers of the concepts included in the concept pairs in the set of concept pairs to the concept analysis model 120, and the concept analysis model 120 may then determine the corresponding feature vectors from the set of constructed feature vectors for training. For example, the concept node with serial number 0 may correspond to the feature vector corresponding to the first row of the matrix, the concept node with serial number 1 may correspond to the feature vector corresponding to the second row of the matrix, and so on.

[0051] As previously mentioned, the concept analysis model 120 can be a classification model. In this case, the classification function of the concept analysis model 120 is to classify a concept (e.g., concept A) into another related concept (e.g., concept B). That is, with concept A as input, the concept analysis model 120 outputs concept B. Therefore, the higher the correlation between concept A and concept B, the more accurate the classification result. Combined with the above description, it can be considered that classifying a concept into its subordinate concepts indicates that the classification result is more accurate.

[0052] Since the concept analysis model 120 constructs a set of feature vectors in a multidimensional vector space according to a predefined number of dimensions, the feature vectors in the set of feature vectors have a relationship in a classification sense.

[0053] According to an embodiment of the present disclosure, when the concept analysis model 120 updates the parameter set of the concept analysis model 120 based at least on a set of concept pairs, the concept pairs in the set of concept pairs received by the concept analysis model 120 from the model parameter updating device 110 can refer to standard classification inputs and classification outputs. For example, when the concept analysis model 120 is applied with the concept pair (professional actor 404, voice actor 405) for the concept node "professional actor 404", it means that the concept "professional actor 404" should be classified as "voice actor 405". Since the concepts "professional actor 404" and "voice actor 405" in the form of feature vectors initially generated by the concept analysis model 120 are likely to be unable to achieve this classification effect, the concept analysis model 120 will adjust at least a portion of the feature vectors in the set of generated feature vectors so that it can transform the feature vector corresponding to "professional actor 404" into the feature vector corresponding to "voice actor 405" in the vector space through the classification operation, that is, classify "professional actor 404" as "voice actor 405". That is, when a concept pair is applied to the concept analysis model 120, the desired result is to increase the probability of the first concept in the concept pair being classified as the second concept in the concept pair through the concept analysis model 120, that is, to increase the measure of the first concept in the concept pair matching the second concept in the concept pair through the concept analysis model 120, where the measure can be a probability or the number of successful assignments.

[0054] Since the concept pairs in the set of concept pairs applied by the model parameter updating device 110 to the concept analysis model 120 clearly indicate the classification input and classification output, the concept system 101 is not important from the perspective of training the concept analysis model 120. In fact, many times, the applied concept pairs cannot reflect the hierarchical relationship between the first concept and the second concept, and the concept analysis model 120 does not need to know the hierarchical relationship between the first concept and the second concept. Therefore, the feature vectors in the set of feature vectors trained by the concept analysis model 120 themselves do not reflect the hierarchical relationship between the concepts corresponding to the feature vectors. In theory, the tree structure of the concept system 101 can be restored by using the set of feature vectors trained by the concept analysis model 120 in a partial order, but the training of the concept analysis model 120 is not carried out for this purpose.

[0055] According to an embodiment of the present disclosure, the model parameter updating device 110 can apply concept pairs in a set of concept pairs to the concept analysis model 120 at various levels. Various application forms including sequential application or batch application can be adopted, wherein the concept pairs input in each batch can be used at the same time to train the feature vectors involved in the concepts therein. Theoretically, if all concept pairs can be applied to the concept analysis model 120 at one time, the concept analysis model 120 can simultaneously train the classification inputs and classification outputs indicated by these concept pairs, so that the training results obtained will be better, that is, the optimization direction will be the most correct. However, considering the trade-offs among the computing power, computing cost, required computing time and computing efficiency of the concept analysis model 120 and the model parameter updating device 110, it is usually most cost-effective to utilize the form of batch application, wherein the number of concept pairs in each batch can be adjusted according to the aforementioned factors. When the concept analysis model 120 is trained, the training of the feature vectors corresponding to the concepts involved in the concept pairs applied later will be based on the feature vectors that have been trained, that is, the training of the feature vectors corresponding to the concepts involved in the concept pairs applied earlier may have caused many feature vectors to be adjusted, and the training of the feature vectors corresponding to the concepts involved in the concept pairs applied later will further adjust the feature vectors that have been adjusted before.

[0056] When the concept analysis model 120 is a classification model, after training, when a concept (i.e., the serial number corresponding to the concept) is input into the concept analysis model 120, the output will be the probability distribution of the concept being classified into all other concepts in the concept system 101. Since the probability of concepts with low correlation directly or indirectly forming concept pairs is small, there will be fewer such concept pairs applied to the concept analysis model 120. Therefore, after training, the probability of the concept analysis model 120 classifying these two concepts into each other will also be small, that is, when two concepts are very unrelated, the probability of classifying them into each other will be very small, and can even approach 0. Therefore, the probability distribution obtained by applying a concept to the concept analysis model 120 can in turn indicate the degree of correlation between this concept and other concepts.

[0057] According to some embodiments of the present disclosure, the training process of the concept analysis model 120 can be gradually carried out by reducing the form of unsuccessful matching (classification). For example, when the concept pair (professional actor 404, voice actor 405) is applied to the concept analysis model 120, if the concept analysis model 120 does not match the concept "voice actor 405" for the concept "professional actor 404", that is, when the concept analysis model 120 does not classify the concept "professional actor 404" as the concept "voice actor 405", it means that a matching failure has occurred. The training goal of the concept analysis model 120 can be defined to reduce the matching failure.

[0058] According to some embodiments of the present disclosure, the model optimization objective of the training process of the concept analysis model 120 can be described as maximizing the probability P(train(nodeC)|nodeC), that is, for a certain concept, maximizing the probability of the training sample set (train(nodeC)) that classifies the concept into this concept through the concept analysis model 120.

[0059] It should be understood that the updating process of the concept analysis model 120 shown in block 520 may be iteratively implemented according to the requirements of the training results of the concept analysis model 120 .

[0060] In box 530, the model parameter updating device 110 provides the threshold matching degree, the threshold number of updates and / or other appropriate preset values ​​to the concept analysis model 120, so that the concept analysis model 120 determines whether to continue updating the parameter set based on these preset values. In some embodiments, if the aforementioned metric (i.e., the metric by which the first concept in a concept pair matches the second concept in this concept pair through the concept analysis model 120) exceeds the threshold matching degree, the concept analysis model 120 may stop updating the parameter set in response thereto. Alternatively or additionally, the concept analysis model 120 may stop updating the parameter set when the number of times the parameter set is updated exceeds the threshold number of updates. For example, when the threshold number of updates is 3, if the concept analysis model 120 has updated the parameter set 3 times, the parameter set is stopped from being updated.

[0061] In some embodiments, the model parameter updating device 110 may provide the concept analysis model 120 with, for example, Figure 1 The training end condition 115 shown in FIG. 1 indicates when the concept analysis model 120 stops updating (training) the parameter set (the set of feature vectors). The training end condition 115 can be embodied in the form of a threshold matching degree and a threshold number of updates. When the training end condition 115 is embodied as a threshold matching degree, when the measure of the first concept in the concept pair matching the second concept in the concept pair by the concept analysis model 120 exceeds the threshold matching degree, the concept analysis model 120 can stop continuing to update the set of feature vectors.

[0062] It should be understood that the aforementioned metric exceeding the threshold matching degree can also be reflected as the number of unsuccessful matches occurring becoming less than a certain threshold. When the training end condition 115 is reflected as a threshold update number, when the number of updates of the set of feature vectors by the concept analysis model 120 reaches the threshold update number, the concept analysis model 120 can stop updating the set of feature vectors. In summary, when the training end condition 115 is met, it can be considered that the training results of the concept analysis model 120 are converging, and at this time, the updating of the set of feature vectors can be stopped.

[0063] It should be understood that block 530 is an optional step in method 500 . When the concept analysis model 120 itself includes the training end condition 115 , the step shown in block 530 may not be required without affecting the implementation of the embodiments of the present disclosure.

[0064] Compared to conventional approaches, the aforementioned embodiments of the present disclosure utilize model parameter updating device 110 to embed the tree-like hierarchical generalization structure of concept system 101 into a vector space via concept analysis model 120, thereby obtaining a vectorized concept representation, i.e., a set of feature vectors, that reflects the hierarchical similarity of concepts within concept system 102. Using the embodiments of the present disclosure, similarity information in the concept hierarchy can be introduced for each concept word in text processing tasks, thereby enriching the feature representation of the concept word and thereby improving the effectiveness of the text processing algorithm.

[0065] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 7 FIG. 7 is a schematic block diagram of an apparatus 700 for updating model parameters according to some embodiments of the present disclosure. The apparatus 700 may be used in, for example, Figure 1 The model parameter updating device 110 is implemented. Figure 7 As shown, the apparatus 700 may include a concept pair set determining module 710 , a concept pair set applying module 720 , and an optional stop updating condition providing module 730 .

[0066] In some embodiments, the concept pair set determination module 710 can be configured to determine a set of concept pairs for training the concept analysis model 120 based on the associations between concepts in the predefined concept system 101, wherein each concept pair in the set of concept pairs includes a first concept and a second concept associated with the first concept. The operation of the concept pair set determination module 710 is similar to that of the above-mentioned method in conjunction with Figure 5 The operations described in block 520 will not be described in detail here.

[0067] According to some embodiments of the present disclosure, the first set determination unit 710 may include (not shown in the figure): a concept subset determination module, configured to determine a subset of concepts corresponding to each concept in the concept system 101, and the concepts in the subset of concepts have a high correlation with the corresponding concepts; and a concept pairing module, configured to pair each concept in the concept with a concept in the corresponding subset of concepts, respectively, as a concept pair in the set of concept pairs. The operation of the concept subset determination module and the concept pairing module is similar to the above combination. Figure 6 The operations described in blocks 610 and 620 are not repeated here.

[0068] In some embodiments, the concept pair set application module 720 can be configured to apply the set of concept pairs to the concept analysis model 120, so that the concept analysis model 120 updates the parameter set of the concept analysis model based on at least the set of concept pairs, thereby improving the measure of the first concept of each concept pair in the set of concept pairs matching the second concept through the concept analysis model. In some embodiments, the concept pair set application module 720 may include (not shown in the figure): a feature vector information providing module, which is configured to provide the number and dimension of feature vectors to be generated to the concept analysis model 120 before applying the set of concept pairs to the concept analysis model 120, so that the concept analysis model 120 generates a set of feature vectors based on the number and dimension. The operation of the concept pair set application module 720 and the feature vector information providing module is similar to the above-mentioned combination. Figure 5 The operations described in block 520 will not be described in detail here.

[0069] According to some embodiments of the present disclosure, the stop updating condition providing module 730 may be configured to provide at least one of a threshold matching degree and a threshold updating number to the concept analysis model 120, so that the concept analysis model 120 stops updating the parameter set in response to at least one of the following: the metric exceeds the threshold matching degree; and the number of times the parameter set is updated exceeds the threshold updating number. The operation of the stop updating condition providing module 730 is similar to the above-mentioned combination of Figure 5 The operations described in block 530 will not be described in detail here.

[0070] It should be understood that each module described in the apparatus 700 is respectively Figure 5 and Figure 6 Each step in the methods 500 and 600 described above corresponds to each other. Moreover, the operations and features of the apparatus 700 and the modules contained therein correspond to those described above in conjunction with Figure 5 and Figure 6 The operations and features described above are the same and have the same effects, and the specific details are not repeated here.

[0071] The modules included in the device 700 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more modules can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the modules in the device 700 can be implemented at least in part by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0072] Figure 7 The modules shown in the figure may be partially or entirely implemented as hardware modules, software modules, firmware modules, or any combination thereof. In particular, in some embodiments, the processes, methods, or procedures described above may be implemented by hardware in a storage system, a host corresponding to the storage system, or other computing devices independent of the storage system.

[0073] Figure 8 FIG2 shows a schematic block diagram of an exemplary computing device 800 that can be used to implement an embodiment of the present disclosure. The device 800 can be used to implement Figure 1 As shown in the figure, the device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 802 or computer program instructions loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The CPU 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0074] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0075] The processing unit 801 performs the various methods and processes described above, such as methods 500 and 600. For example, in some embodiments, methods 500 and 600 may be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the CPU 801, one or more steps of the methods 500 and 600 described above may be performed. Alternatively, in other embodiments, the CPU 801 may be configured to perform methods 500 and 600 in any other suitable manner (e.g., via firmware).

[0076] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0077] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0078] In addition, although adopting specific order to describe each operation, this should be understood as requiring such operation to be carried out in the specific order shown or in sequential order, or requiring that all illustrated operations should be carried out to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although comprising some specific implementation details in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation also can be implemented in a plurality of implementations individually or in the mode of any suitable subcombination.

[0079] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for updating model parameters, comprising: determining, based on the associations between semantic concepts in a predefined semantic concept system, a set of concept pairs for training a concept analysis model, each concept pair in the set of concept pairs comprising a first concept and a second concept associated with the first concept, wherein a hierarchical structure of concepts in the concept system represents a semantic structure of the concepts; and Applying the set of concept pairs to the concept analysis model so that the concept analysis model updates a parameter set of the concept analysis model based at least on the set of concept pairs, thereby improving a measure of matching of the first concept with the second concept by the concept analysis model for each concept pair in the set of concept pairs; The set of concept pairs determined for training the concept analysis model includes: determining a subset of concepts corresponding to each of the concepts from the concept system, the concepts in the subset of concepts having a high correlation with the corresponding concept; and pairing each of the concepts with a corresponding concept in the subset of concepts as a concept pair in the set of concept pairs; The subset of concepts identified includes: According to the most similar node set of each node and the node's hierarchical relationship, a partial order relationship of the similarity between each node and other nodes in the concept system is constructed, where the most similar node set of each node is all its descendant nodes; Based on the partial order relationship of the similarity between each node and other nodes in the concept system, the subset of concepts corresponding to each concept is determined; The measure of whether the first concept matches the second concept through the concept analysis model includes: the probability of obtaining the corresponding second concept as output through the concept analysis model with the first concept as input.

2. The method of claim 1 , wherein the parameter set comprises a set of feature vectors, the method further comprising: Before applying the set of concept pairs to the concept analysis model, the number and dimension of feature vectors to be generated are provided to the concept analysis model, so that the concept analysis model generates the set of feature vectors based on the number and dimension.

3. The method according to claim 1, further comprising: At least one of a threshold matching degree and a threshold update number is provided to the concept analysis model so that the concept analysis model stops updating the parameter set in response to at least one of: The metric exceeds the threshold match; and The parameter set is updated a number of times exceeding the threshold update number.

4. The method according to claim 1, wherein the association comprises at least one of the following: the hierarchical relationships between the concepts; and The logical relationship between the concepts described.

5. A device for updating model parameters, comprising: a concept pair set determination module configured to determine a set of concept pairs for training a concept analysis model based on the associations between semantic concepts in a predefined semantic concept system, wherein each concept pair in the set of concept pairs includes a first concept and a second concept associated with the first concept, wherein a hierarchical structure of concepts in the concept system represents a semantic structure of the concepts; and a concept pair set application module configured to apply the set of concept pairs to the concept analysis model, so that the concept analysis model updates a parameter set of the concept analysis model based at least on the set of concept pairs, thereby improving a measure of matching of the first concept with the second concept of each concept pair in the set of concept pairs through the concept analysis model; The concept pair set determination module includes: a concept subset determination module configured to determine, from the concept system, a subset of concepts corresponding to each of the concepts, wherein the concepts in the subset of concepts have a high correlation with the corresponding concept; and a concept pairing module configured to pair each of the concepts with a corresponding concept in the subset of concepts as a concept pair in the set of concept pairs; The concept subset determination module includes: The relationship construction module is configured to construct a partial order relationship of the similarity between each node and other nodes in the concept system based on the most similar node set of each node and the node's hierarchical relationship, where the most similar node set of each node is all its descendant nodes; A subset determination module based on a partial order relationship is configured to determine a subset of concepts corresponding to each concept based on a partial order relationship of similarities between each node and other nodes in the concept system; The measure of whether the first concept matches the second concept through the concept analysis model includes: the probability of obtaining the corresponding second concept as output through the concept analysis model with the first concept as input.

6. The apparatus according to claim 5, wherein the parameter set comprises a set of feature vectors, the apparatus further comprising: The feature vector information providing module is configured to provide the concept analysis model with the number and dimension of feature vectors to be generated before applying the set of concept pairs to the concept analysis model, so that the concept analysis model generates the set of feature vectors based on the number and the dimension.

7. The apparatus according to claim 5, further comprising: A stop-update condition providing module is configured to provide at least one of a threshold matching degree and a threshold update number to the concept analysis model, so that the concept analysis model stops updating the parameter set in response to at least one of the following: The metric exceeds the threshold match; and The parameter set is updated a number of times exceeding the threshold update number.

8. The apparatus according to claim 5, wherein the association comprises at least one of the following: the hierarchical relationships between the concepts; and The logical relationship between the concepts described.

9. An electronic device comprising: one or more processors; as well as A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 4 when executed by a processor.

Images