Method, device, medium and program product for storing language model

Abstract

The disclosed embodiments disclose a method, device, medium and program product for storing a language model, the method comprising: obtaining a to-be-stored vocabulary list, the to-be-stored vocabulary list comprising a plurality of word sequences and probability information of each word sequence, the word sequences comprising a high-order word sequence and a low-order word sequence, each high-order word sequence comprising at least one low-order word sequence; determining a parent-child subordinate relationship between high-order word sequences and low-order word sequences; based on the parent-child subordinate relationship, determining a tree structure corresponding to the to-be-stored vocabulary list, the nodes in the tree structure corresponding to the word sequences one-to-one; when a node corresponding to a word sequence in the tree structure is a non-leaf node, generating state information of the word sequence based on the probability information of the word sequence and the probability information of the word sequence corresponding to a child node of the non-leaf node; determining a mapping relationship between the word sequence and the state information; based on the parent-child subordinate relationship and the mapping relationship, storing the to-be-stored vocabulary list and the state information.

Description

Technical Field

[0001] The present disclosure relates to a method for storing a language model, an electronic device, a storage medium, and a computer program product. Background Art

[0002] In the field of natural language, language models are an important part of speech recognition systems and directly affect the accuracy of speech recognition systems. Generally speaking, the larger the language model, the higher the accuracy of speech recognition, but at the same time, the more memory space it occupies, which leads to many limitations of larger language models in practical applications. For example, in speech recognition systems, constructing a decoding graph is a step that requires a large amount of memory space. The memory usage of constructing a decoding graph is proportional to the size of the language model, and its peak memory usage can reach hundreds of times the size of the language model itself. This leads to the direct use of a language model to construct a decoding graph, which often fails due to insufficient memory.

[0003] In order to avoid the above phenomenon, in the related art, a two-pass decoding method is usually adopted to use a large language model. Specifically, a decoding graph is first constructed by a small language model, and a decoding is performed based on the decoding graph. Then, the decoding grid obtained by the one-pass decoding is re-scored by the large language model, i.e., a two-pass decoding. In this way, only the large language model needs to be read into the memory, and the large language model is avoided to be directly used to construct the decoding graph. Summary of the invention

[0004] Embodiments of the present disclosure provide a method, electronic device, storage medium, and computer program product for storing a language model to reduce the space occupied by the language model in speech recognition codec in memory.

[0005] One aspect of an embodiment of the present disclosure provides a method, comprising: obtaining a to-be-stored vocabulary list, the to-be-stored vocabulary list comprising a plurality of word sequences and probability information of each word sequence, wherein the word sequence comprises a high-order word sequence and a low-order word sequence, and each high-order word sequence comprises at least one low-order word sequence; determining a parent-child subordinate relationship between the high-order word sequence and the low-order word sequence; based on the parent-child subordinate relationship, determining a tree structure corresponding to the to-be-stored vocabulary list, the nodes in the tree structure corresponding one-to-one to the word sequence; when a node corresponding to the word sequence in the tree structure is a non-leaf node, generating state information of the word sequence based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child node of the non-leaf node; determining a mapping relationship between the word sequence and the state information; based on the parent-child subordinate relationship and the mapping relationship, storing the to-be-stored vocabulary list and the state information.

[0006] In some embodiments, based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child node of the non-leaf node, the state information of the word sequence is generated, including: based on the parent-child affiliation, determining the word sequence corresponding to the child node of the non-leaf node, and constructing a data pair for the word sequence corresponding to each child node, the data pair including numbering data and information data, and determining the last bit of the information data as a leaf identification bit; if the child node is a non-leaf child node, then the number and pointer of the word sequence corresponding to the non-leaf child node are written into the numbering data and information data in the data pair corresponding to the non-leaf child node, respectively, and the leaf identification bit of the data pair is set to 1, and the pointer is used to determine the storage address of the state information of the word sequence corresponding to the non-leaf child node; if the child node is a leaf child node, then the number and probability information of the word sequence corresponding to the child node are written into the numbering data and information data in the data pair corresponding to the child node, respectively, and the leaf identification bit of the data pair is set to 0, wherein the probability information of the leaf child node does not include fallback probability; storing the data pair as an array element in a preset sequence container; generating state information based on the probability information of the word sequence corresponding to the non-leaf node and the calling instruction of the sequence container.

[0007] In some embodiments, a pointer is generated via the following steps: determining the address offset of the storage location of the state information of the word sequence corresponding to the non-leaf child node relative to the storage location of the state information of the word sequence corresponding to the non-leaf child node; and determining the address offset as a pointer to the word sequence corresponding to the non-leaf child node.

[0008] In some embodiments, the method also includes: determining the first bit of the information data as an overflow flag; if the number of non-leaf child nodes is less than the storage space of the information data, setting the overflow flag of the data pair of the non-leaf child node to 0; if the number of non-leaf child nodes is not less than the storage space of the information data, storing the storage address of the state information of the word sequence corresponding to the non-leaf child node as an array element in a pre-set overflow array; setting the overflow flag in the data pair of the non-leaf child node to 1; and determining the sequence number of the storage address of the state information of the word sequence corresponding to the non-leaf child node in the overflow array as a pointer to the word sequence corresponding to the non-leaf child node.

[0009] In some embodiments, the method further includes: if the word sequence corresponding to the leaf node in the tree structure is a lowest-order word sequence, then generating state information of the lowest-order word sequence based on probability information of the lowest-order word sequence.

[0010] In some embodiments, the method further includes: storing the storage address of the state information of the lowest-order word sequence as an array element in a pre-constructed one-dimensional array.

[0011] In some embodiments, storing status information includes: sorting the status information of the word sequence corresponding to the child nodes of the non-leaf nodes, and storing the status information of the word sequence corresponding to the child nodes of the non-leaf nodes in sequence into storage locations adjacent to the status information of the word sequence corresponding to the non-leaf nodes according to the sorting results.

[0012] In some embodiments, the method further includes: in response to an instruction to determine the language model probability of the target sentence, segmenting the target sentence to obtain a segmentation set consisting of lowest-order word sequences; determining the probability of each lowest-order word sequence in the segmentation set based on the status information of each lowest-order word sequence in the segmentation set; if the segmentation set includes only one lowest-order word sequence, determining the probability of the lowest-order word sequence as the language model probability of the target sentence; if the segmentation set includes more than one lowest-order word sequence, arranging each lowest-order word sequence in the segmentation set according to semantics to obtain at least one high-order word sequence; if at least one high-order word sequence is a leaf node, determining the probability of at least one high-order word sequence based on the status information of the word sequence corresponding to the parent node of the leaf node; if at least one high-order word sequence is a non-leaf node, determining the probability of at least one high-order word sequence based on the status information of at least one high-order word sequence; determining the language model probability of the target sentence based on the probabilities of each lowest-order word sequence in the segmentation set and the probability of at least one high-order word sequence.

[0013] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, wherein when the computer program is executed, the method for storing a language model in any of the above embodiments is implemented.

[0014] Another aspect of the embodiments of the present disclosure provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for storing a language model in any of the above embodiments is implemented.

[0015] According to another aspect of the embodiments of the present disclosure, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the method for storing a language model in any of the above embodiments.

[0016] The method for storing a language model provided by the embodiment of the present disclosure can obtain the word list to be stored, and can determine the parent-child affiliation between the high-order word sequence and the low-order word sequence according to the sorting of the low-order word sequence contained in the high-order word sequence, and determine the tree structure corresponding to the word list to be stored based on the parent-child affiliation; when the node corresponding to the word sequence in the tree structure is a non-leaf node, based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child node of the non-leaf node, generate the state information of the word sequence; then, determine the mapping relationship between the word sequence and the state information; finally, based on the parent-child affiliation and the mapping relationship, store the word list to be stored and the state information. It can avoid generating state information for the word sequence corresponding to the leaf node, reduce the content of redundant data in the language model, and thus reduce the memory usage of the language model.

[0017] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0020] Figure 1 is a schematic diagram of a fragment of an exemplary language model file;

[0021] Figure 2 A flowchart of an embodiment of a method for storing a language model disclosed in the present invention;

[0022] Figure 3 A flowchart of generating state information in one embodiment of a method for storing a language model disclosed herein;

[0023] Figure 4 A flowchart of another embodiment of the method for storing a language model disclosed herein;

[0024] Figure 5 A flowchart of another embodiment of the method for storing a language model disclosed herein;

[0025] Figure 6 This is a schematic diagram of the structure of an embodiment of a device for storing a language model disclosed in the present invention;

[0026] Figure 7 The figure is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0027] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0028] Those skilled in the art will understand that the terms “", “” and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0029] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0030] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0031] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0032] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0033] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0034] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0035] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0036] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0037] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0038] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0039] In the process of implementing the present disclosure, the inventors found that although the current two-pass decoding method can avoid directly constructing a decoding graph through a large language model, thereby avoiding decoding failure due to insufficient memory, when the large language model increases to a certain extent, even if the large language model is only read into the memory, it will take up a considerable amount of memory. Therefore, a language model that is still too large cannot be used.

[0040] The memory usage of the language model is related to the size of the language model itself on the one hand, and to the storage structure of the language model on the other hand. In the related art, there is a lot of redundant data in the storage structure of the language model. For example, each word sequence in the vocabulary has a corresponding state information, which is used to store the probability information of the word sequence (for example, it may include conditional probability and fallback probability) and the probability information of the child nodes of the node corresponding to the word sequence in the tree storage structure (for example, it may include conditional probability). Since there are no child nodes in the leaf nodes, the probability information of the word sequence corresponding to the leaf nodes only has conditional probability or language model probability, but no fallback probability. For such nodes, it is only necessary to store its conditional probability or language model probability. If the probability information of the above leaf nodes is stored with state information, it will cause a waste of memory. When the number of leaf nodes is large, it will lead to a large amount of redundant data.

[0041] Generally, a method for calculating a language model probability through a language model includes the following process: For example, it is necessary to calculate a model probability P(S) of a sentence "ABCD", where A, B, C, and D represent four words respectively.

[0042] Then P(S)=P(A)*P(AB)*P(ABC)*P(ABCD), where P(A) represents the language model probability of A; P(AB) represents the language model probability of the high-order word sequence (AB) composed of A and B, that is, the conditional probability that the first word is A and the current word is B; P(ABC) represents the language model probability of a higher-order word sequence (ABC) composed of a high-order word sequence (AB) and C, that is, the conditional probability that the first two words are A and B and the current word is C; P(ABCD) represents the language model probability of a higher-order word sequence (ABCD) composed of a high-order word sequence (ABC) and D, that is, the conditional probability that the first three words are A, B, C and the current word is D.

[0043] If a conditional probability does not exist in the language model, for example, if P(ABCD) does not exist, then fallback will occur: P(ABCD) = P(BCD)*BP(ABC), where BP(ABC) represents the fallback probability of the high-order word sequence (ABC). If the fallback probability does not exist, then the fallback probability is equal to 1. If P(BCD) does not exist either, then P(BCD) will continue to fallback to P(CD)*BP(BC), and so on recursively until the lowest-order word sequence (for example, the language model probability of A, B, C, D). Until the above probability value P(S) is calculated.

[0044] Combine the following Figure 1 For example, Figure 1 A fragment of an exemplary language model file is shown. Figure 1 In the example shown, the language model is a 3-gram model, and the language model file 100 is in Arpa format, which is a standard format for n-gram models.

[0045] After obtaining the vocabulary to be stored, the electronic device (for example, a server) for storing the 3-gram model can parse the language model file 100 to determine the word sequences of each order in the vocabulary to be stored and the probability information of each word sequence. Figure 1 110 is a set of first-order word sequences (i.e., 1-gram), and 140 is the probability information corresponding to the first-order word sequences in the set; 120 is a set of second-order word sequences (i.e., 2-gram), and 150 is the probability information corresponding to the second-order word sequences in the set; 130 is a set of third-order word sequences (i.e., 3-gram), and 150 is the probability information corresponding to the second-order word sequences in the set.

[0046] By comparing the order word sequences, it can be seen that the high-order word sequence includes at least one low-order word sequence. For example, the second-order word sequence 170 is composed of two first-order word sequences, and the third-order word sequence 190 is composed of three first-order word sequences. The execution subject can determine the tree structure corresponding to the vocabulary to be stored by determining the parent-child subordinate relationship between the high-order word sequence and the low-order word sequence. For example, the first second-order word sequence contained in the third-order word sequence 190 is 170, then the third-order word sequence 190 is subordinate to the second-order word sequence 170, and at the same time, the first first-order word sequence contained in the second-order word sequence 170 is "today", then the second-order word sequence 170 is subordinate to the first-order word sequence "today". By mapping this subordinate relationship to the tree structure, the parent-child relationship between each node can be determined. The node corresponding to the third-order word sequence 190 is the child node of the node corresponding to the second-order word sequence 170, and the node corresponding to the second-order word sequence 170 is the child node of the node corresponding to the first-order word sequence "today".

[0047] Afterwards, the execution subject can determine the leaf nodes and non-leaf nodes in the tree structure according to whether there are child nodes. Leaf nodes refer to nodes without child nodes in the tree structure, and non-leaf nodes refer to nodes with child nodes in the tree structure. For example, each node corresponding to the third-order word sequence set 130 has no child nodes, so each node corresponding to the third-order word sequence set 130 is a leaf node; the node corresponding to the second-order word sequence 180 also has no child nodes, so the node corresponding to the second-order word sequence 180 is also a leaf node; the node corresponding to the second-order word sequence 170 has a child node (the node corresponding to the third-order word sequence 190), so the node corresponding to the second-order word sequence 170 is a non-leaf node.

[0048] If and only if the node corresponding to the word sequence in the tree structure is a non-leaf node, the state information of the word sequence is generated based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child node of the non-leaf node. Figure 1 Taking the second-order word sequence 170 in the example as an example, the execution entity can generate the state information of the second-order word sequence 170 (specifically, it can be lm_state in the Arpa format) based on the probability information corresponding to the second-order word sequence 170 (for example, it can include conditional probability and fallback probability) and the probability information corresponding to the third-order word sequence 190 (for example, conditional probability).

[0049] The state information thus obtained only includes the state information of the word sequence corresponding to the non-leaf node. For the word sequence corresponding to the leaf node, no state information is generated, which can effectively reduce the amount of state information and remove the redundant data brought about by it.

[0050] Then, the execution subject determines the mapping relationship between the word sequence and the state information, and stores the word list and state information to be stored based on the parent-child relationship and the mapping relationship. This can reduce the content of redundant data in the language model, thereby reducing the memory usage of the language model.

[0051] Exemplary Methods

[0052] The following is an exemplary description of the method for storing a language model disclosed in the present invention.

[0053] First reference Figure 2 , Figure 2 A flowchart of an embodiment of a method for storing a language model in the present disclosure is shown, and the flowchart includes the following steps:

[0054] Step 210: Obtain a vocabulary to be stored.

[0055] In this embodiment, the word list to be stored includes multiple word sequences and probability information of each word sequence, the word sequences include high-order word sequences and low-order word sequences, and each high-order word sequence contains at least one low-order word sequence.

[0056] Generally, the probability information of a word sequence includes the following three types of probabilities: language model probability, fallback probability, and conditional probability. The probability information of different types of word sequences may include one or two of them, such as Figure 1 As shown, in the 3-gram model, the probability information of some lowest-order word sequences (i.e., 1-gram) may include language model probability and fallback probability, while the probability information of some lowest-order word sequences only includes language model probability; the probability information of third-order word sequences (i.e., 3-gram) only includes conditional probability; the probability information of some second-order word sequences (i.e., 2-gram) only includes conditional probability, while the probability information of some second-order word sequences includes conditional probability and fallback probability.

[0057] It should be noted that the high-order word sequence and the low-order word sequence in this embodiment can be relative concepts. If word sequence 1 contains word sequence 2, then word sequence 1 is a high-order word sequence relative to word sequence 2, and correspondingly, word sequence 2 is a low-order word sequence; if word sequence 2 contains word sequence 3, then word sequence 2 is a high-order word sequence relative to word sequence 3, and correspondingly, word sequence 3 is a low-order word sequence.

[0058] Step 220: Determine the parent-child relationship between the high-order word sequence and the low-order word sequence.

[0059] In this embodiment, the execution subject may extract word sequences of different orders from the list to be stored, and then determine the parent-child subordinate relationship between word sequences of different orders.

[0060] As an example, the execution entity may first determine the first (m-1)-order word sequence contained in the m-order word sequence, and then determine that the m-order word sequence is subordinate to the (m-1)-order word sequence; further, the execution entity may determine the first (m-2)-order word sequence contained in the (m-1)-order word sequence, and then determine that the (m-1)-order word sequence is subordinate to the (m-2)-order word sequence, where m is a positive integer greater than 2.

[0061] Step 230: Based on the parent-child relationship, determine the tree structure corresponding to the vocabulary to be stored.

[0062] Among them, the nodes in the tree structure correspond one to one to the word sequences.

[0063] Usually, in order to obtain a faster search speed, a tree structure is usually used to store the language model. The tree structure can use nodes to represent word sequences, and use the parent-child relationship between nodes to represent the parent-child relationship between word sequences of different orders.

[0064] For example, the node corresponding to the (m-1)-order word sequence in step 220 is the parent node of the node corresponding to the m-order word sequence, and is also a child node of the (m-2)-order word sequence.

[0065] Step 240: When the node corresponding to the word sequence in the tree structure is a non-leaf node, based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child nodes of the non-leaf node, generate the state information of the word sequence.

[0066] In this embodiment, the probability information in the state information includes both the probability information of the word sequence corresponding to the non-leaf node and the probability information of the word sequence corresponding to the child node of the non-leaf node.

[0067] Combined with the n-gram language model, the second-order word sequence E (i.e., 2-gram) and the third-order word sequence F (i.e., 3-gram) have a parent-child subordinate relationship, and the node corresponding to the second-order word sequence E is a non-leaf node. The execution subject can generate the state information of E based on the probability information of E and F. Among them, the probability information of E includes conditional probability and fallback probability. When F is a leaf node, the probability information of F only includes conditional probability, but not fallback probability. When F is a non-leaf node, the probability information of F includes both conditional probability and fallback probability.

[0068] In a specific example of the embodiment, a fragment of the status information of E in the Arpa file is as follows:

[0069]

[0070] Among them, LmState_E represents the state information of E, logprob_ and backoff_logprob_ represent the conditional probability and backoff probability of E respectively; the children field stores the probability information of F. Specifically, an array is pre-stored in the unordered map (unordered_map), and each element in the array represents the mapping relationship between a 32-bit integer and LmState_F*. The integer represents the number of F in the vocabulary, and LmState_F* is the address pointer storing the probability information of F.

[0071] Step 250: Determine the mapping relationship between the word sequence and the state information.

[0072] In this embodiment, the mapping relationship may represent the corresponding relationship between the word sequence and the state information.

[0073] Step 260: based on the parent-child relationship and the mapping relationship, store the vocabulary to be stored and the state information.

[0074] In this embodiment, the parent-child subordinate relationship can represent the tree storage structure of the vocabulary, and the execution subject can store the word sequences of each order in the vocabulary according to the parent-child subordinate relationship. The mapping relationship can represent the corresponding relationship between the word sequence and the state information, and when the execution subject runs the language model, the probability information of each word sequence in the vocabulary can be determined according to the mapping relationship.

[0075] The method for storing a language model provided by the embodiment of the present disclosure can obtain the word list to be stored, and can determine the parent-child affiliation between the high-order word sequence and the low-order word sequence according to the sorting of the low-order word sequence contained in the high-order word sequence, and determine the tree structure corresponding to the word list to be stored based on the parent-child affiliation; when the node corresponding to the word sequence in the tree structure is a non-leaf node, based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child node of the non-leaf node, generate the state information of the word sequence; then, determine the mapping relationship between the word sequence and the state information; finally, based on the parent-child affiliation and the mapping relationship, store the word list to be stored and the state information. It can avoid generating state information for the word sequence corresponding to the leaf node, reduce the content of redundant data in the language model, and thus reduce the memory capacity of the language model.

[0076] Then refer to the reference Figure 3 , Figure 3 A flowchart of another embodiment of the present disclosure for storing a language model is shown. Figure 3 As shown, the process includes the following steps:

[0077] Step 310: Obtain a vocabulary to be stored.

[0078] Step 320: Determine the parent-child relationship between the high-order word sequence and the low-order word sequence.

[0079] Step 330: Based on the parent-child relationship, determine the tree structure corresponding to the vocabulary to be stored. In this embodiment, steps 310 to 330 correspond to the aforementioned steps 210 to 230, which will not be described in detail here.

[0080] Step 340: Based on the parent-child subordinate relationship, determine the word sequence corresponding to the child node of the non-leaf node, and construct a data pair for the word sequence corresponding to each child node. The data pair includes numbering data and information data, and the last bit of the information data is determined as the leaf identification bit.

[0081] In this embodiment, the leaf flag is used to identify whether the child node is a leaf child node.

[0082] In a specific example, the word sequence corresponding to non-leaf node 1 is G, and its child nodes include node 2 and node 3, which correspond to word sequences H and J respectively. The execution subject can use the pair() function to construct data pairs for word sequences H and J respectively. The specific exemplary code can be std::pair<int32,int32> , where the first 32-bit integer (int32) is used to store the number of H or J in the vocabulary, the first 31 bits of the second 32-bit integer (int32) are used to store the probability information of H or J, and the 32nd bit is the leaf identification bit.

[0083] Step 350: If the child node is a non-leaf child node, the number and pointer of the word sequence corresponding to the non-leaf child node are written into the number data and information data in the data pair corresponding to the non-leaf child node, respectively, and the leaf identification bit of the data pair is set to 1.

[0084] In this embodiment, the pointer is used to determine the storage address of the state information of the word sequence corresponding to the non-leaf child node, for example, the pointer can point to the real address of the state information of the word sequence corresponding to the child node in the memory. The leaf flag bit value of 1 indicates that the child node is a non-leaf child node.

[0085] Continuing with the example in step 340, if node 2 is a non-leaf child node, the number of H in the vocabulary is written into std::pair<int32,int32> In the first 32-bit integer in , write the pointer of H's probability information into the first 31 bits of the second 32-bit integer, and then set the 32nd bit to 1. When the language model needs to obtain the probability information of H, it can first extract the data pair from the numerical value, then extract the pointer of H from the second 32-bit integer of the data pair, and finally determine the storage location of H's probability information based on the pointer.

[0086] In some optional implementations of the present embodiment, the following steps may be used to generate a pointer to a word sequence: determine the address offset of the storage location of the state information of the word sequence corresponding to the non-leaf child node relative to the storage location of the state information of the word sequence corresponding to the non-leaf node; and determine the address offset as the pointer to the word sequence corresponding to the non-leaf child node.

[0087] In this implementation, the address offset between the storage locations of the probability information of two word sequences is used as a pointer, which can reduce the pointer length, so that the information data in the data pair can store a larger number of pointers.

[0088] Step 360: If the child node is a leaf child node, the number and probability information of the word sequence corresponding to the child node are written into the number data and information data in the data pair corresponding to the child node, respectively, and the leaf flag of the data pair is set to 0.

[0089] In this embodiment, since a leaf node has no child nodes, the probability information of the word sequence corresponding to the leaf node only includes the conditional probability, but not the fallback probability. A leaf flag bit of 0 indicates that the child node is a leaf node.

[0090] Continuing with the example in step 340, if node 3 is a leaf child node, the number of J in the vocabulary is written into std::pair<int32,int32> In the first 32-bit integer in , write the probability information (conditional probability) of J into the first 31 bits of the second 32-bit integer, and then set the 32nd bit to 0. When the language model needs to obtain the probability information of J, it can first extract the data pair from the array, and then divide the value of the second 32-bit integer in the data pair by 2 (equivalent to bitwise shifting one bit right and taking the value of the first 31 bits) to obtain the probability information (i.e. conditional probability) of J.

[0091] Step 370: store the data pairs as array elements into a preset sequence container.

[0092] In this embodiment, the sequence container (for example, std::vector) pre-stores an array whose size can be changed dynamically, and the storage position of each element in the array is continuous. In this way, when the language model obtains the probability information of the word sequence, it can access the elements in the array through the offset on the pointer.

[0093] Step 380: Generate status information based on the probability information of the word sequence corresponding to the non-leaf node and the calling instruction of the sequence container.

[0094] In conjunction with the example in step 340 and the following exemplary code, the execution subject may write the calling instruction of the sequence container into the children field in the state information to obtain the state information LmState_G of G.

[0095]

[0096] Among them, LmState_G represents the state information of G, logprob_g represents the conditional probability of G, backoff_logprob_g represents the backoff probability of G, std::vector <std::pair<int32,int32> >Indicates the calling instruction of the sequence container that stores the data pair of H and J.

[0097] Step 390: Determine the mapping relationship between the word sequence and the state information.

[0098] Step 391: store the vocabulary to be stored and the state information based on the parent-child subordinate relationship and the mapping relationship.

[0099] It should be noted that step 390 and step 391 correspond to the aforementioned step 260 and step 270 respectively, and will not be repeated here.

[0100] and Figure 2 Compared to the embodiment shown, Figure 3 The embodiment shown embodies the steps of combining the number and probability information of the word sequence into data pairs and storing the data pairs in a sequence container. The sequence container can be used to store each data pair in a continuous position, which can further reduce the memory usage of the language model on the one hand, and dynamically adjust the size of the array on the other hand, which helps to improve access and storage efficiency.

[0101] In practice, when the number of word sequences is large, the pointer length of the word sequence may exceed the storage space of the information data, that is, the number of probability information of the word sequence overflows. For example, when the information data is a 32-bit integer, the pointer length is greater than 31 bits. In order to avoid this extreme situation, the method for storing a language model disclosed in the present invention may also include Figure 4 The process shown. Figure 4 A flowchart of another embodiment of the present disclosure for storing a language model is shown. Figure 4 As shown, Figure 3 Based on the process shown, the method may further include the following steps:

[0102] Step 410: determine the first bit of the information data as the overflow flag bit.

[0103] In this embodiment, the value of the overflow flag is used to identify whether the amount of probability information of the word sequence overflows.

[0104] As an example, the execution body can use std::pair<int32,int32> The first bit of the second 32-bit integer in is determined as the overflow flag bit, so that only the middle 30 bits of the second 32-bit integer are used to store the probability information of the word sequence.

[0105] Step 420: If the number of non-leaf child nodes is less than the storage space for information data, set the overflow flag of the data pair of the non-leaf child node to 0.

[0106] In this embodiment, the overflow flag is 0, indicating that the number of non-leaf child nodes has not overflowed. At this time, the storage space of the information data is larger than the pointer length of the probability information of the word sequences corresponding to all non-leaf child nodes, that is, the storage space of the information data meets the storage requirements.

[0107] Step 430: If the number of non-leaf child nodes is not less than the storage space for information data, the storage address of the state information of the word sequence corresponding to the non-leaf child node is used as an array element and stored in a pre-set overflow array, and the overflow flag in the data pair of the non-leaf child node is set to 1.

[0108] In this embodiment, the overflow flag is 1, indicating that the number of non-leaf child nodes overflows. At this time, the storage space of the information data is less than or equal to the pointer length of the probability information of the word sequence corresponding to all non-leaf child nodes, that is, the storage space of the information data does not meet the storage requirements.

[0109] Step 440: determine the serial number of the storage address of the state information of the word sequence corresponding to the non-leaf child node in the overflow array as the pointer of the word sequence corresponding to the non-leaf child node.

[0110] In this embodiment, the serial number of the element in the overflow array may represent the offset of the storage position of the element relative to the storage position of the array.

[0111] In a specific example, when the language model needs to obtain the probability information of the word sequence corresponding to the non-leaf child node, it can first determine the data pair corresponding to the word sequence from the sequence container, then read the information data in the data pair, and extract the pointer of the word sequence from the information data. It can be understood that the leaf identification bit of the information data is 1 at this time. If the overflow identification bit of the information data is 1, then the pointer is the serial number of the state information of the word sequence in the overflow array, and then the element pointed to by the serial number is determined from the overflow array, and the storage location of the state information of the word sequence is read from the element. If the overflow identification bit of the information data is 0, then the pointer can represent the storage location or offset of the state information of the word sequence, and then the storage location of the state information of the word sequence can be determined according to the pointer. Finally, the probability information of the word sequence is extracted from the state information.

[0112] In this embodiment, when the number of non-leaf child nodes overflows, the storage address of the state information of the word sequence corresponding to the non-leaf child node can be stored as an array element in a preset overflow array, so that more addresses can be stored, which can be suitable for language models with a large number of word sequences.

[0113] In practice, when the language model is applied, the frequency of searching and accessing the lowest order word sequence is high, and accordingly, the frequency of parsing the state information of the lowest order word sequence is also high, resulting in a large amount of calculation. Taking this into account, in some optional implementations of any of the above embodiments, the method may also include: if the word sequence corresponding to the leaf node in the tree structure is the lowest order word sequence, then based on the probability information of the lowest order word sequence, generating the state information of the lowest order word sequence.

[0114] In this implementation, state information can be generated for the lowest order word sequence corresponding to the leaf node, and the state information of the lowest order word sequence only includes the language model probability of the lowest order word sequence, but does not include the fallback probability. In this way, each lowest order word sequence corresponds to a state information, and the language model can directly read the probability information of the lowest order word sequence from the state information when applied, which can avoid the increase in the amount of calculation caused by frequent parsing of the state information of the lowest order word sequence and improve the running speed.

[0115] Furthermore, the method may further include: storing the storage address of the state information of the lowest order word sequence as an array element in a pre-constructed one-dimensional array. In this way, the storage efficiency of the state information of the lowest order word sequence can be improved to prevent the state information of the lowest order word sequence from occupying too much memory.

[0116] In some optional implementations of any of the above embodiments, storing status information includes: sorting the status information of the word sequences corresponding to the child nodes of the non-leaf nodes, and storing the status information of the word sequences corresponding to the child nodes of the non-leaf nodes in sequence into storage locations adjacent to the status information of the word sequences corresponding to the non-leaf nodes according to the sorting results.

[0117] In this implementation, the status information of the word sequence corresponding to the child nodes of the non-leaf node is sequentially stored in the storage locations adjacent to the status information of the word sequence corresponding to the non-leaf node. This can reduce the offset of the storage location of the status information of the word sequence corresponding to the child nodes of the non-leaf node relative to the status information of the word sequence corresponding to the non-leaf node, help reduce the length of the pointer, and thus improve storage capacity.

[0118] Next reference Figure 5 , Figure 5 A flowchart of an embodiment of the present disclosure for storing a language model is shown. Based on the above embodiment, the method may also include: Figure 5 The process shown includes:

[0119] Step 510: In response to the instruction to determine the language model probability of the target sentence, the target sentence is segmented to obtain a segmentation set consisting of a lowest-order word sequence.

[0120] In this embodiment, after the execution subject reads the language model into the memory, when receiving an instruction to calculate the language model probability of the target sentence, the target sentence can be divided into several words by a word segmentation tool to obtain a word segmentation set.

[0121] As an example, the target sentence is "How is the weather in Beijing?", and the segmentation set obtained after the execution subject performs segmentation on the target sentence includes: "Beijing", "weather", and "how is it".

[0122] Step 520: Determine the probability of each lowest-order word sequence in the word segmentation set.

[0123] In this embodiment, for the lowest order word sequence corresponding to a non-leaf node, the execution subject can directly read the probability of the lowest order word sequence from the state information. For the lowest order word sequence corresponding to a leaf node, the execution subject can extract the probability of the lowest order word sequence from the word list to be stored.

[0124] In some optional implementations of this embodiment, if the lowest-order word sequence corresponding to the leaf node also has state information, the probability of the lowest-order word sequence can be directly read from the state information.

[0125] Step 530: If the word segmentation set includes only one lowest-order word sequence, the probability of the lowest-order word sequence is determined as the language model probability of the target sentence.

[0126] Step 540: If the word segmentation set includes more than one lowest-order word sequence, arrange the lowest-order word sequences in the word segmentation set according to semantics to obtain at least one high-order word sequence.

[0127] With reference to the example in step 510 , the words in the word segmentation set may be combined into the following high-order word sequence: “Beijing weather”, “How is the weather in Beijing”, “How is the weather”.

[0128] Step 550: If at least one high-order word sequence is a leaf node, determine the probability of at least one high-order word sequence based on the state information of the word sequence corresponding to the parent node of the leaf node.

[0129] In this embodiment, the word sequence corresponding to the leaf node does not have its own corresponding status information. The execution entity can first determine the parent node of the leaf node based on the parent-child subordinate relationship, and then parse the probability information of the word sequence corresponding to the leaf node from the status information of the word sequence corresponding to the parent node to obtain the probability of the leaf node.

[0130] Step 560: If at least one high-order word sequence is a non-leaf node, determine the probability of at least one high-order word sequence based on the state information of the at least one high-order word sequence.

[0131] In this embodiment, the execution subject can directly parse the probability information of the word sequence corresponding to the non-leaf node from the state information to obtain the probability of the word sequence corresponding to the non-leaf node.

[0132] Step 570: Determine the language model probability of the target sentence based on the probability of each lowest-order word sequence in the word segmentation set and the probability of at least one high-order word sequence.

[0133] In a specific example, the target sentence is "How is the weather in Beijing?". The word segmentation set obtained after the execution subject segments the target sentence includes: "Beijing", "weather", and "how". The high-order word sequence obtained by semantically sorting these three words includes: "Beijing weather", "How is the weather in Beijing?"

[0134] With P representing probability and BP representing fallback probability, the language model probability of the target sentence can be expressed as: P = P(Beijing)*P(Beijing weather)*P(how is the weather in Beijing). If "Beijing weather" or "how is the weather in Beijing" does not exist in the vocabulary, fallback occurs. At this time, the model language probability of the target sentence is:

[0135] P = P(Beijing)*P(weather)*BP(Beijing)*P(how is the weather)*BP(Beijing weather). Since "Beijing weather" does not exist, BP(Beijing weather) = 1, then the model language probability of the target sentence is:

[0136] P=P(Beijing)*P(weather)*BP(Beijing)*P(how is the weather)

[0137] Afterwards, the execution entity can first obtain the language model probabilities of "Beijing" and "weather", and then determine the conditional probability of "how is the weather" by parsing the status information of "weather", and then determine the fallback probability of "Beijing" by parsing the status information of "Beijing". Finally, the obtained probability values ​​are multiplied to determine the language model probability of the target sentence.

[0138] In this embodiment, the language model only includes the status information of non-leaf nodes, which can increase the amount of computation of the probability information of the search word sequence and improve the computational efficiency of the language model.

[0139] Exemplary Devices

[0140] like Figure 6 As shown, another aspect of the embodiment of the present disclosure provides an apparatus for storing a language model, including: a vocabulary acquisition unit 610, configured to acquire a vocabulary to be stored, the vocabulary to be stored includes multiple word sequences and probability information of each word sequence, wherein the word sequence includes a high-order word sequence and a low-order word sequence, and each high-order word sequence contains at least one low-order word sequence; a subordinate relationship determination unit 620, configured to determine the parent-child subordinate relationship between the high-order word sequence and the low-order word sequence; a tree structure determination unit 630, configured to determine the tree structure to be stored based on the parent-child subordinate relationship A tree structure corresponding to the word list, wherein the nodes in the tree structure correspond one-to-one to the word sequence; a state information generating unit 640 is configured to: when the node corresponding to the word sequence in the tree structure is a non-leaf node, generate the state information of the word sequence based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child node of the non-leaf node; a mapping relationship determining unit 650 is configured to determine the mapping relationship between the word sequence and the state information; a model storing unit 660 is configured to store the word list to be stored and the state information based on the parent-child subordinate relationship and the mapping relationship.

[0141] In some embodiments, the state information generation unit 640 further includes: a data pair construction module, which is configured to: determine the word sequence corresponding to the child node of the non-leaf node based on the parent-child subordinate relationship, and construct a data pair for the word sequence corresponding to each child node, the data pair includes number data and information data, and the last bit of the information data is determined as a leaf identification bit; a writing module, which is configured to: if the child node is a non-leaf child node, write the number and pointer of the word sequence corresponding to the non-leaf child node into the number data and information data in the data pair corresponding to the non-leaf child node, respectively, and set the leaf identification bit of the data pair to 1, and the pointer is used to determine The storage address of the status information of the word sequence corresponding to the non-leaf child node; the writing module is configured to: if the child node is a leaf child node, write the number and probability information of the word sequence corresponding to the child node into the number data and information data in the data pair corresponding to the child node respectively, and set the leaf identification bit of the data pair to 0, wherein the probability information of the word sequence corresponding to the child node does not include the fallback probability; the data pair storage module is configured to store the data pair as an array element into a preset sequence container; the information generation module is configured to generate status information based on the probability information of the word sequence corresponding to the non-leaf node and the calling instruction of the sequence container.

[0142] In some embodiments, the device also includes a pointer generation unit configured to: determine the address offset of the storage location of the state information of the word sequence corresponding to the non-leaf child node relative to the storage location of the state information of the word sequence corresponding to the non-leaf child node; and determine the address offset as a pointer to the word sequence corresponding to the non-leaf child node.

[0143] In some embodiments, the device also includes: an overflow identification unit, configured to determine the first bit of the information data as an overflow identification bit; an overflow setting unit, configured to set the overflow identification bit of the data pair of the non-leaf child node to 0 if the number of non-leaf child nodes is less than the storage space of the information data; the overflow setting unit is configured to: if the number of non-leaf child nodes is not less than the storage space of the information data, store the storage address of the state information of the word sequence corresponding to the non-leaf child node as an array element in a pre-set overflow array; set the overflow identification bit in the data pair of the non-leaf child node to 1; and determine the sequence number of the storage address of the state information of the word sequence corresponding to the non-leaf child node in the overflow array as a pointer to the word sequence corresponding to the non-leaf child node.

[0144] In some embodiments, the device also includes: a lowest-order word sequence unit, configured to: if a leaf node in the tree structure is a lowest-order word sequence, generate state information of the lowest-order word sequence based on probability information of the lowest-order word sequence.

[0145] In some embodiments, the apparatus further comprises: an array construction unit configured to store the storage address of the lowest-order word sequence as an array element into a pre-constructed one-dimensional array.

[0146] In some embodiments, the model storage unit 660 further includes: a state information storage module, configured to: sort the state information of the word sequence corresponding to the child nodes of the non-leaf nodes, and store the state information of the word sequence corresponding to the child nodes of the non-leaf nodes in sequence into a storage location adjacent to the state information of the word sequence corresponding to the non-leaf nodes according to the sorting result.

[0147] In some embodiments, the device further includes: a word segmentation unit configured to segment the target sentence in response to an instruction to determine the language model probability of the target sentence, and obtain a word segmentation set consisting of a lowest-order word sequence; a probability acquisition unit configured to determine the probability of each lowest-order word sequence in the word segmentation set based on the state information of each lowest-order word sequence in the word segmentation set; a first probability determination unit configured to, if the word segmentation set includes only one lowest-order word sequence, determine the probability of the lowest-order word sequence as the language model probability of the target sentence; a second probability determination unit configured to: if the word segmentation set includes more than one lowest-order word sequence, arrange each lowest-order word sequence in the word segmentation set according to semantics to obtain at least one high-order word sequence; if at least one high-order word sequence is a leaf node, determine the probability of at least one high-order word sequence based on the state information of the word sequence corresponding to the parent node of the leaf node; if at least one high-order word sequence is a non-leaf node, determine the probability of at least one high-order word sequence based on the state information of at least one high-order word sequence; and determine the language model probability of the target sentence based on the probabilities of each lowest-order word sequence in the word segmentation set and the probability of at least one high-order word sequence.

[0148] In addition, an embodiment of the present disclosure further provides an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program stored in the memory, and when the computer program is executed, the method for storing a language model in any of the above embodiments of the present disclosure is implemented.

[0149] Figure 7 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. Figure 7 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the device and the device, or a stand-alone device independent of them, and the stand-alone device may communicate with the device and the device to receive the collected input signals from them.

[0150] like Figure 7 As shown, the electronic device includes one or more processors and memory.

[0151] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0152] The memory may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the method for storing a language model and / or other desired functions of the various embodiments of the present disclosure described above.

[0153] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0154] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0155] The output device can output various information to the outside, including the determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0156] Of course, to simplify, Figure 7 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0157] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the method for storing a language model according to various embodiments of the present disclosure described in the above part of this specification.

[0158] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0159] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, cause the processor to execute the steps of the method for storing a language model according to various embodiments of the present disclosure described in the above part of this specification.

[0160] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0161] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0162] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0163] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0164] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0165] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0166] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0167] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0168] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for storing a language model, characterized in that: include: Acquire a to-be-stored word list in a to-be-stored language model, wherein the to-be-stored word list includes a plurality of word sequences and probability information of each of the word sequences, wherein the word sequences include high-order word sequences and low-order word sequences, and each of the high-order word sequences includes at least one of the low-order word sequences; Determining a parent-child relationship between the high-order word sequence and the low-order word sequence; Based on the parent-child subordinate relationship, determining a tree structure corresponding to the to-be-stored vocabulary, wherein the nodes in the tree structure correspond one-to-one to the word sequence; When the node corresponding to the word sequence in the tree structure is a non-leaf node, based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child node of the non-leaf node, generating the state information of the word sequence; Determine a mapping relationship between the word sequence and the state information; Based on the parent-child affiliation and the mapping relationship, storing the vocabulary to be stored and the state information; Based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child node of the non-leaf node, the state information of the word sequence is generated, including: If the child node is a non-leaf child node, the number and pointer of the word sequence corresponding to the non-leaf child node are written into the number data and information data in the data pair corresponding to the non-leaf child node, and the leaf identification bit of the data pair is set to 1. The pointer is used to determine the storage address of the status information of the word sequence corresponding to the non-leaf child node; If the child node is a leaf child node, the number and probability information of the word sequence corresponding to the child node are written into the number data and information data in the data pair corresponding to the child node, respectively, and the leaf flag of the data pair is set to 0, wherein the probability information of the word sequence corresponding to the child node does not include the fallback probability; Storing the data pairs as array elements into a preset sequence container; The state information is generated based on the probability information of the word sequence corresponding to the non-leaf node and the calling instruction of the sequence container.

2. The method according to claim 1, characterized in that Based on the probability information of the word sequence and the probability information of the word sequence corresponding to the child node of the non-leaf node, the state information of the word sequence is generated, which also includes: Based on the parent-child subordinate relationship, the word sequence corresponding to the child node of the non-leaf node is determined, and a data pair is constructed for the word sequence corresponding to each child node, the data pair includes the numbering data and the information data, and the last bit of the information data is determined as the leaf identification bit.

3. The method according to claim 2, characterized in that The pointer is generated by the following steps: Determine the address offset of the storage location of the state information of the word sequence corresponding to the non-leaf child node relative to the storage location of the state information of the word sequence corresponding to the non-leaf node; The address offset is determined as a pointer to the word sequence corresponding to the non-leaf child node.

4. The method according to claim 2, characterized in that: The method further comprises: Determine the first bit of the information data as an overflow flag bit; If the number of the non-leaf child nodes is less than the storage space of the information data, the overflow flag bit of the data pair of the non-leaf child node is set to 0; If the number of the non-leaf child nodes is not less than the storage space of the information data, the storage address of the status information of the word sequence corresponding to the non-leaf child node is used as an array element and stored in a preset overflow array; the overflow flag in the data pair of the non-leaf child node is set to 1; and the sequence number of the storage address of the status information of the word sequence corresponding to the non-leaf child node in the overflow array is determined as a pointer to the word sequence corresponding to the non-leaf child node.

5. The method according to claim 1, characterized in that The method further comprises: If the word sequence corresponding to the leaf node in the tree structure is a lowest-order word sequence, then based on the probability information of the lowest-order word sequence, the state information of the lowest-order word sequence is generated.

6. The method according to claim 5, characterized in that The method further comprises: The storage address of the state information of the lowest-order word sequence is stored as an array element in a pre-constructed one-dimensional array.

7. The method according to any one of claims 1 to 6, characterized in that Storing the state information includes: The state information of the word sequence corresponding to the child nodes of the non-leaf node is sorted, and the state information of the word sequence corresponding to the child nodes of the non-leaf node is sequentially stored in a storage position adjacent to the state information of the word sequence corresponding to the non-leaf node according to the sorting result.

8. The method according to any one of claims 1 to 6, characterized in that The method further comprises: In response to an instruction to determine a language model probability of a target sentence, segmenting the target sentence to obtain a segmentation set consisting of a lowest-order word sequence; Determining the probability of each lowest-order word sequence in the word segmentation set; If the word segmentation set includes only one lowest-order word sequence, determining the probability of the lowest-order word sequence as the language model probability of the target sentence; If the word segmentation set includes more than one lowest-order word sequence, the lowest-order word sequences in the word segmentation set are arranged according to semantics to obtain at least one high-order word sequence; if the at least one high-order word sequence is a leaf node, the probability of the at least one high-order word sequence is determined based on the state information of the word sequence corresponding to the parent node of the leaf node; if the at least one high-order word sequence is a non-leaf node, the probability of the at least one high-order word sequence is determined based on the state information of the at least one high-order word sequence; based on the probabilities of the lowest-order word sequences in the word segmentation set and the probability of the at least one high-order word sequence, the language model probability of the target sentence is determined.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute a computer program stored in the memory, and when the computer program is executed, implement the method described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method described in any one of claims 1 to 8 is implemented.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method described in any one of claims 1 to 8 is implemented.

Images