Semantic recognition model training method, device, electronic device and storage medium

Abstract

The present application discloses a training method, device, electronic device and storage medium for a semantic recognition model, and belongs to the field of Internet technology. The method obtains a sample search term and a first sample business information, determines the masking methods corresponding to the multiple fields of the first sample business information based on the field types corresponding to the multiple fields of the first sample business information, and masks the words in the multiple fields of the first sample business information based on the masking methods corresponding to the multiple fields to obtain the second sample business information, and performs model training based on the sample search term and the second sample business information to obtain a semantic recognition model. Since the words in multiple fields are masked, the model can fully learn the semantic information of multiple fields, thereby improving the reasoning ability of the semantic recognition model.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a training method, device, electronic device and storage medium for a semantic recognition model. Background Art

[0002] With the development of Internet technology, search services have been widely used. In search services, semantic relevance can be used to measure the semantic relevance between the user's search terms and the business information returned by the electronic device. Therefore, the process of determining semantic relevance can be converted into a process of semantic matching between search terms and business information. At present, the matching degree between search terms and business information is mainly determined through semantic recognition models.

[0003] In the related technology, when training a semantic recognition model, the main step is to obtain sample search terms and sample business information, and then randomly mask some words in the sample business information to obtain the masked sample business information. Then, based on the sample search terms, the masked words in the masked sample business information are predicted, and the model is trained based on the prediction accuracy to obtain a semantic recognition model.

[0004] However, sample business information can generally be divided into multiple fields with clear semantic information. According to the random masking method in related technologies, it may happen that words in some fields are completely masked, while words in other fields are not masked. This will cause the model to be unable to fully learn the semantic information of each field in the sample business information, resulting in poor reasoning ability of the semantic recognition model. Summary of the invention

[0005] The present application embodiment provides a method, device, electronic device and storage medium for training a semantic recognition model, which can improve the reasoning ability of the semantic recognition model. The technical solution is as follows:

[0006] On the one hand, a method for training a semantic recognition model is provided, the method comprising:

[0007] Obtaining sample search terms and first sample business information;

[0008] Determining, based on field types respectively corresponding to multiple fields of the first sample service information, masking modes respectively corresponding to multiple fields of the first sample service information, wherein the multiple fields are obtained by dividing the first sample service information based on semantic information, and one field corresponds to one masking mode;

[0009] Based on masking modes corresponding to the multiple fields of the first sample business information, respectively, masking the words in the multiple fields of the first sample business information to obtain the second sample business information;

[0010] Model training is performed based on the sample search terms and the second sample business information to obtain a semantic recognition model.

[0011] On the one hand, a training device for a semantic recognition model is provided, the device comprising:

[0012] An acquisition module, used to acquire a sample search term and first sample business information;

[0013] A first determining module, configured to determine masking modes corresponding to the multiple fields of the first sample service information based on field types corresponding to the multiple fields of the first sample service information, wherein the multiple fields are obtained by dividing the first sample service information based on semantic information, and one field corresponds to one masking mode;

[0014] a masking module, configured to mask the words in the multiple fields of the first sample service information respectively based on the masking modes respectively corresponding to the multiple fields of the first sample service information, so as to obtain the second sample service information;

[0015] The training module is used to perform model training based on the sample search terms and the second sample business information to obtain a semantic recognition model.

[0016] On the one hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement a training method for a semantic recognition model such as any possible implementation method described above.

[0017] On the one hand, a computer-readable storage medium is provided, in which at least one program code is stored. The at least one program code is loaded and executed by a processor to implement a training method for a semantic recognition model such as any possible implementation method described above.

[0018] On the one hand, a computer program product is provided, wherein at least one program code is stored in the computer program product, and the at least one program code is loaded and executed by a processor to implement a method for training a semantic recognition model such as any possible implementation method described above.

[0019] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0020] An embodiment of the present application provides a method for training a semantic recognition model. When training the semantic recognition model, the method first divides the first sample business information into multiple fields, and then masks the words in the multiple fields respectively according to the field types corresponding to the multiple fields. Since the words in the multiple fields are masked, the model can fully learn the semantic information of the multiple fields, thereby improving the reasoning ability of the semantic recognition model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 It is a schematic diagram of an implementation environment of a method for training a semantic recognition model provided in an embodiment of the present application;

[0023] Figure 2 It is a flowchart of a method for training a semantic recognition model provided in an embodiment of the present application;

[0024] Figure 3 It is a flowchart of a method for training a semantic recognition model provided in an embodiment of the present application;

[0025] Figure 4 is a schematic diagram of a training semantic recognition model provided in an embodiment of the present application;

[0026] Figure 5 It is a schematic diagram of the correlation scores corresponding to the semantic recognition model provided by the related art and the semantic recognition model provided by the present application;

[0027] Figure 6 It is a structural schematic diagram of a training device for a semantic recognition model provided in an embodiment of the present application;

[0028] Figure 7 It is a structural block diagram of a terminal provided in an embodiment of the present application;

[0029] Figure 8 It is a structural block diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0031] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0032] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the search terms and business information involved in this application are all obtained with full authorization.

[0033] Figure 1 Schematic diagram of an implementation environment of a semantic recognition model training method provided in an embodiment of the present application. Figure 1 The implementation environment includes: an electronic device, which can be provided as a terminal 101, can be provided as a server 102, or can be provided as a terminal 101 and a server 102. If the electronic device is provided as a terminal 101 and a server 102, the terminal 101 and the server 102 are connected via a wireless or wired network. In the embodiments of the present application, the electronic device is not specifically limited.

[0034] If the electronic device is provided as a terminal 101, the terminal 101 performs model training to obtain a semantic recognition model. The semantic recognition model can be deployed in the terminal 101, so that the terminal 101 can recognize search terms through the semantic recognition model and search for corresponding business information based on the search terms.

[0035] If the electronic device is provided as a server 102, the server 102 performs model training to obtain a semantic recognition model. The semantic recognition model can be deployed in the terminal 101. Accordingly, the implementation environment also includes: the terminal 101, the terminal 101 and the server 102 can be connected via a wireless or wired network, the terminal 101 can identify the search word through the semantic recognition model, and search for corresponding business information based on the search word.

[0036] If the electronic device is provided as a terminal 101 and a server 102, the server 102 performs model training to obtain a semantic recognition model, and then the semantic recognition model is deployed to the terminal 101, so that the terminal 101 can recognize search terms through the semantic recognition model and search for corresponding business information based on the search terms.

[0037] The terminal 101 is at least one of a mobile phone, a tablet computer, a PC (Personal Computer), etc. The server 102 can be at least one of a server, a server cluster consisting of multiple servers, a cloud server, a cloud computing platform, and a virtualization center.

[0038] One point that needs to be explained is that the semantic recognition model provided in the embodiment of the present application is a pre-trained semantic recognition model. Before the pre-trained semantic recognition model is deployed in the terminal 101, it needs to be further trained based on labeled sample search terms and sample business information to obtain a target semantic recognition model. The target semantic recognition model can be deployed in the terminal 101, so that the terminal 101 recognizes the search terms through the target semantic recognition model and searches for corresponding business information based on the search terms.

[0039] In an embodiment of the present application, if the search term is used to search for food, the terminal can search for the merchant corresponding to the food through the target semantic recognition model. If the search term is used to search for documents, the terminal can search for documents including the search term through the target semantic recognition model. If the search term is used to search for clothing, the terminal can search for clothing matching the search term through the target semantic recognition model. The terminal can also search for other scenarios through the target semantic recognition model, which are not listed here one by one.

[0040] Figure 2 is a flowchart of a method for training a semantic recognition model provided in an embodiment of the present application, see Figure 2 , the method comprising:

[0041] Step 201: Obtain sample search terms and first sample business information.

[0042] Step 202: Based on the field types corresponding to the multiple fields of the first sample business information, determine the masking methods corresponding to the multiple fields of the first sample business information, where the multiple fields are obtained by dividing the first sample business information based on semantic information, and one field corresponds to one masking method.

[0043] Step 203: Based on the masking modes corresponding to the multiple fields of the first sample business information, respectively mask the words in the multiple fields of the first sample business information to obtain the second sample business information.

[0044] Step 204: Perform model training based on the sample search terms and the second sample business information to obtain a semantic recognition model.

[0045] An embodiment of the present application provides a method for training a semantic recognition model. When training the semantic recognition model, the method first divides the first sample business information into multiple fields, and then masks the words in the multiple fields respectively according to the field types corresponding to the multiple fields. Since the words in the multiple fields are masked, the model can fully learn the semantic information of the multiple fields, thereby improving the reasoning ability of the semantic recognition model.

[0046] Figure 3 is a flowchart of a method for training a semantic recognition model provided in an embodiment of the present application, which is executed by an electronic device, see Figure 3 , the method comprising:

[0047] Step 301: The electronic device obtains a sample search term and first sample business information.

[0048] The sample search term may be any search term in a search scenario, and the first sample business information is business information searched based on the sample search term.

[0049] For example, if the sample search term is a search term for searching for food, the first sample business information may be the name of the merchant corresponding to the food, the address of the merchant, the brand and category of the food, etc. For another example, if the sample search term is a search term for searching for documents, the first sample business information may be the document containing the search term, the title of the document, the category of the document, etc., which is searched based on the search term.

[0050] Among them, the number of sample search terms and the number of first sample business information can be set and changed as needed, and there is no specific limitation on this in the embodiment of the present application.

[0051] In the embodiment of the present application, after the electronic device obtains the first sample business information, it can divide the first sample business information based on the semantic information to obtain multiple fields, and perform step 302 based on the multiple fields. The multiple fields may include a title field, a brand field, a category field, an address field, a keyword field, etc., and may also include other types of fields, which are not listed here one by one.

[0052] One thing that needs to be explained is that after the electronic device obtains the sample search term and the first sample business information, it can directly execute step 302, or it can discard some other fields in the first sample business information except the title field, and then combine the remaining other fields and the sample search term to form the third sample business information, use the third sample business information as the first sample business information, and then execute step 302.

[0053] The process of the electronic device discarding some other fields can be: the electronic device randomly selects a fifth preset proportion of other fields from multiple other fields except the title field of the first sample business information, discards the fifth preset proportion of other fields, and splices the title field and the remaining other fields together to form the third sample business information.

[0054] The electronic device may splice the title field and the remaining other fields together in a certain order, or may splice the title field and the remaining other fields together at will, which is not specifically limited here. In addition, the fifth preset ratio may be set and changed as needed, for example, the fifth preset ratio is 10%, 15% or 20%, and the fifth preset ratio of 15% is used as an example for explanation here.

[0055] In an embodiment of the present application, the title field is the most important field in the first sample business information. When other fields are missing, the title field can be used to directly match the sample search term. Therefore, before masking, some fields except the title field can be randomly discarded to ensure that the title field is learned more fully.

[0056] Step 302: The electronic device determines masking modes corresponding to the multiple fields of the first sample service information based on the field types corresponding to the multiple fields of the first sample service information.

[0057] One field corresponds to a masking method, which can be a word-level masking method or an alternating masking method, wherein the alternating masking method is used to indicate that the word in the field is masked by alternating the word-level masking method and the word-level masking method. Masking in the embodiment of the present application refers to masking the selected word by using a masking mark.

[0058] The character-level masking method refers to masking a character in a field, and the word-level masking method refers to masking a phrase in a field. For example, if the field is "hot pot", the character-level masking method is to mask the character "fire" or the character "pot", and the word-level masking method is to mask the phrase "hot pot".

[0059] The multiple fields of the first sample business information may include a title field, a brand field, a category field, a keyword field, an address field, and other types of fields, which are not specifically limited here. The electronic device determines that the field type corresponding to the title field is a title field type, the field type corresponding to the brand field is a brand field type, the field type corresponding to the category field is a category field type, the field type corresponding to the keyword field is a keyword field type, and the field corresponding to the address field is an address field type.

[0060] Based on the correspondence between field types and masking methods, the electronic device determines the masking method corresponding to the title field type, the masking method corresponding to the keyword field type, the masking method corresponding to the category field type, the masking method corresponding to the brand field type, and the masking method corresponding to the address field type. Among them, the masking method corresponding to the title field type, the masking method corresponding to the keyword field type, and the masking method corresponding to the category field type are all word-level masking methods, and the masking method corresponding to the brand field type and the masking method corresponding to the address field type are all alternating masking methods. It can be seen that one field type corresponds to one masking method, but the same masking method can be applied to multiple field types.

[0061] If the first sample business information also includes other types of fields except the title field, the brand field, the category field, the keyword field and the address field, the electronic device determines that the masking method of the other types of fields is the alternating masking method.

[0062] It should be noted that if the electronic device first discards some fields other than the title field in the first sample service information and then executes step 302, the first sample service information in step 302 is service information consisting of the title field and the remaining other fields.

[0063] Step 303: the electronic device masks the words in the multiple fields of the first sample service information respectively based on the masking modes respectively corresponding to the multiple fields of the first sample service information, to obtain the second sample service information.

[0064] The masking methods include a word-level masking method and an alternating masking method. The alternating masking method is used to indicate that the word-level masking method and the word-level masking method are alternately used to mask the words in the field.

[0065] In a possible implementation, for any field in the first sample service information, if the masking mode corresponding to the field is an alternating masking mode, the electronic device masks the words in the field through the following steps (A-1) to (A-2):

[0066] (A-1) If the masking method corresponding to the field is an alternating masking method, and based on the first probability, it is determined that a word-level masking method is currently used for the words in the field, the electronic device selects a first target word from the field and performs word-level masking on the first target word.

[0067] Since the alternating masking method alternately adopts the word-level masking method and the character-level masking method, before masking, the electronic device needs to determine whether the word-level masking method or the character-level masking method is currently adopted. Different masking methods correspond to different probabilities. Therefore, the electronic device can determine whether the word-level masking method or the character-level masking method is currently adopted based on the first probability and the second probability. Among them, the first probability is used to represent the probability of performing word-level masking on the words in the field, and the second probability is used to represent the probability of performing word-level masking on the words in the field.

[0068] The process of an electronic device determining whether to use a word-level masking method or a character-level masking method based on a first probability and a second probability can be as follows: the electronic device first determines the probability interval corresponding to the first probability and the probability interval corresponding to the second probability respectively, and then randomly generates a value based on the field. If the probability interval corresponding to the first probability is smaller than the probability interval corresponding to the second probability, and the value is within the probability interval corresponding to the first probability, the electronic device determines that the word-level masking method is currently used. If the probability interval corresponding to the first probability is smaller than the probability interval corresponding to the second probability, and the value is within the probability interval corresponding to the second probability, the electronic device determines that the character-level masking method is currently used. Correspondingly, the probability interval corresponding to the first probability may also be larger than the probability interval corresponding to the second probability. In this case, the electronic device also determines which masking method is currently used based on which probability interval the value randomly generated by the field falls within.

[0069] Among them, the size of the first probability and the second probability can be set and changed as needed, and there is no specific limitation on this. For example, the first probability and the second probability are the same, both are 0.5, the probability interval corresponding to the first probability is 0-0.5, and the probability interval corresponding to the second probability is 0.5-1. The value randomly generated by the electronic device based on the field is 0.4, then the word-level masking method is used for this field.

[0070] In this step, if it is determined based on the first probability that a word-level masking method is currently used for the words in the field, the electronic device selects a first target word from the field and performs word-level masking on the first target word.

[0071] The process of the electronic device selecting the first target word from the field may be: the electronic device determines the first number of phrases included in the field, selects a first preset ratio of phrases from the first number of phrases, and if the number of phrases in the first preset ratio does not exceed the first preset threshold, the electronic device uses the phrases in the first preset ratio as the first target word for word-level masking. If the number of phrases in the first preset ratio exceeds the first preset threshold, the electronic device selects the phrases in the first preset threshold from the phrases in the first preset ratio as the first target word for word-level masking.

[0072] Among them, the electronic device can randomly select a phrase of the first preset threshold from the phrases of the first preset ratio, and the electronic device can also select a phrase of the first preset threshold from the phrases of the first preset ratio in other ways, without specific limitation. In addition, the first preset ratio can be set and changed as needed, without specific limitation. For example, the first preset ratio is 10%, 15% or 20%, and here only the first preset ratio of 15% is used as an example for explanation. The first preset threshold can also be set and changed as needed, without specific limitation.

[0073] (A-2) If the masking method corresponding to the field is an alternating masking method, and based on the second probability, it is determined that a word-level masking method is currently used for the words in the field, the electronic device selects a second target word from the field and performs word-level masking on the second target word.

[0074] In this step, when the electronic device selects the second target word from the field, it determines the fifth number of characters included in the field, selects characters of a sixth preset ratio from the fifth number of characters, and if the number of characters of the sixth preset ratio does not exceed the second preset threshold, the electronic device directly uses the characters of the sixth preset ratio as the second target word for character-level masking. If the number of characters of the sixth preset ratio exceeds the second preset threshold, the electronic device selects characters of the second preset threshold from the characters of the sixth preset ratio as the second target word for character-level masking.

[0075] Among them, the electronic device can randomly select the words of the second preset threshold from the words of the sixth preset ratio, and the electronic device can also select the words of the second preset threshold from the words of the sixth preset ratio in other ways, which is not specifically limited. In addition, the sixth preset ratio can be set and changed as needed, which is not specifically limited. For example, the sixth preset ratio is the same as the first preset ratio, both of which are 15%. The second preset threshold can also be set and changed as needed, which is not specifically limited.

[0076] In the embodiment of the present application, for brand fields, address fields, and other types of fields, the electronic device may mask the words in the fields in an alternating masking manner.

[0077] In another possible implementation, for any field in the first sample business information, if the masking method corresponding to the field is a word-level masking method, and the field type of the field is a category field type, the electronic device performs word-level masking on the words in the field through the following steps (B-1) to (B-3):

[0078] (B-1) If the masking method corresponding to the field is a word-level masking method, the field type of the field is a category field type, and the preset ratio for masking the words in the field is determined to be a second preset ratio based on the third probability, the electronic device selects words of the second preset ratio from the field and performs word-level masking on the words of the second preset ratio.

[0079] The category field can be exhaustively enumerated and has low prediction difficulty. Based on this, the electronic device can use different preset ratios to mask the words in the field. Therefore, before masking the words in the field, it is necessary to determine which preset ratio to use to mask the words in the field.

[0080] Different preset proportions correspond to different probabilities, so the electronic device can determine the preset proportion of words in the field to be masked based on the third probability, the fourth probability and the fifth probability. The third probability is used to indicate the probability of masking the words in the second preset proportion in the field, the fourth probability is used to indicate the probability of masking the words in the third preset proportion in the field, and the fifth probability is used to indicate the probability of masking the words in the fourth preset proportion in the field.

[0081] If the preset proportion for masking the words in the field is determined to be the second preset proportion based on the third probability, the electronic device selects the words of the second preset proportion from the field and performs word-level masking on the words of the second preset proportion.

[0082] In this implementation, the electronic device can randomly select words of the second preset ratio from the field, and then determine whether the number of words of the second preset ratio exceeds the third preset threshold. If the number of words of the second preset ratio does not exceed the third preset threshold, the electronic device directly performs word-level masking on the words of the second preset ratio. If the number of words of the second preset ratio exceeds the third preset threshold, the electronic device randomly selects words of the third preset threshold from the words of the second preset ratio, and performs word-level masking on the words of the third preset threshold.

[0083] (B-2) If the masking method corresponding to the field is a word-level masking method, the field type of the field is a category field type, and the preset ratio is determined to be the third preset ratio based on the fourth probability, the electronic device selects words of the third preset ratio from the field and performs word-level masking on the words of the third preset ratio.

[0084] In this implementation, the electronic device may also randomly select words of the third preset ratio from the field, and then determine whether the number of words of the third preset ratio exceeds the fourth preset threshold. If the number of words of the third preset ratio does not exceed the fourth preset threshold, the electronic device directly performs word-level masking on the words of the third preset ratio. If the number of words of the third preset ratio exceeds the fourth preset threshold, the electronic device randomly selects words of the fourth preset threshold from the words of the third preset ratio, and performs word-level masking on the words of the fourth preset threshold.

[0085] (B-3) If the masking method corresponding to the field is a word-level masking method, the field type of the field is a category field type, and the preset ratio is determined to be the fourth preset ratio based on the fifth probability, the electronic device selects words of the fourth preset ratio from the field and performs word-level masking on the words of the fourth preset ratio.

[0086] In this implementation, the electronic device may also randomly select words of the fourth preset ratio from the field, and then determine whether the number of words of the fourth preset ratio exceeds the fifth preset threshold. If the number of words of the fourth preset ratio does not exceed the fifth preset threshold, the electronic device directly performs word-level masking on the words of the fourth preset ratio. If the number of words of the fourth preset ratio exceeds the fifth preset threshold, the electronic device randomly selects words of the fifth preset threshold from the words of the fourth preset ratio, and performs word-level masking on the words of the fifth preset threshold.

[0087] Among them, the third probability is greater than the fourth probability, and the fourth probability is greater than the fifth probability. On this basis, the third probability, the fourth probability and the fifth probability can be set and changed as needed, without specific limitation. For example, the third probability is 0.5, the fourth probability is 0.4, and the fifth probability is 0.1. In addition, the fourth preset ratio is greater than the third preset ratio, and the third preset ratio is greater than the second preset ratio. On this basis, the second preset ratio, the third preset ratio and the fourth preset ratio can also be set and changed as needed, without specific limitation. For example, the second preset ratio is 15%, the third preset ratio is 50%, and the fourth preset ratio is 100%.

[0088] In another possible implementation, for any field in the first sample business information, if the masking method corresponding to the field is a word-level masking method, and the field type of the field is a title field type, the electronic device performs word-level masking on the words in the field through the following steps (C-1) to (C-2):

[0089] (C-1) If the masking method corresponding to the field is the word-level masking method, the field type of the field is the title field type, and based on the sixth probability, it is determined to select a third target word from the field according to the size of the word weight, the electronic device selects the third target word from the field based on the first weights corresponding to the multiple words included in the field, and performs word-level masking on the third target word, with one word corresponding to one first weight.

[0090] The title field is generally a complete sentence and is relatively long. Based on this, the electronic device can preferentially mask words that are more important to the matching task based on first weights corresponding to multiple words included in the field.

[0091] In an embodiment of the present application, for the title field, the electronic device may select the third target word from the field based on the word weight, or may randomly select the third target word from the field. Different selection methods correspond to different probabilities. Therefore, the electronic device may determine whether to select the third target word from the field based on the word weight or randomly select the third target word from the field based on the sixth probability and the seventh probability. The sixth probability is used to indicate the probability of selecting the third target word according to the size of the word weight, and the seventh probability is used to indicate the probability of randomly selecting the third target word.

[0092] If it is determined based on the sixth probability that the third target word is selected from the field according to the size of the word weight, the electronic device selects the seventh preset proportion of words with the largest first weight from the field based on the first weights corresponding to the multiple words included in the field, and determines whether the number of words in the seventh preset proportion exceeds the sixth preset threshold. If the number of words in the seventh preset proportion does not exceed the sixth preset threshold, the electronic device directly uses the words in the seventh preset proportion as the third target word for word-level masking. If the number of words in the seventh preset proportion exceeds the sixth preset threshold, the electronic device randomly selects words of the sixth preset threshold from the words in the seventh preset proportion, and uses the words of the sixth preset threshold as the third target word for word-level masking.

[0093] It should be noted that before executing step (C-1), the electronic device first determines the first weights corresponding to the multiple words of the first sample service information. This process can be implemented by the following steps (D-1) to (D-4), including:

[0094] (D-1) The electronic device determines the click probabilities corresponding to the first sample business information based on the number of clicks corresponding to the first sample business information.

[0095] The number of clicks is used to indicate the number of times the first sample business information is clicked when searching for the first sample business information through a sample search term. In this step, the number of the sample search terms and the number of the first sample business information are both multiple.

[0096] For any first sample business information, the electronic device may first obtain a click log, count the number of clicks corresponding to the first sample business information under multiple sample search terms from the click log, determine the sum of the number of clicks corresponding to the first sample business information under multiple sample search terms, and obtain the total number of clicks corresponding to the first sample business information. For any sample search term, the electronic device determines the ratio of the number of clicks corresponding to the first sample business information under the sample search term to the total number of clicks, and obtains the click probability of the first sample business information being clicked under the sample search term.

[0097] The electronic device can determine the click probabilities corresponding to the first sample service information according to the method. The electronic device can express the click probabilities corresponding to the first sample service information according to the following formula:

[0098]

[0099] Among them, p ij It is used to indicate the click probability corresponding to the jth first sample business information when searching for the jth first sample business information through the i-th sample search term, that is, the click probability corresponding to the jth first sample business information under the i-th sample search term. Both j and i are integers greater than 0. ij It is used to indicate the number of clicks corresponding to the jth first sample business information under the i-th sample search term, ∑ j click ij Used to indicate the total number of clicks corresponding to the j-th first sample business information.

[0100] (D-2) The electronic device determines a search word set corresponding to the first sample business information based on the click probabilities respectively corresponding to the first sample business information.

[0101] The search word set includes a third number of sample search words, and the first sample business information corresponding to the third number of sample search words has the highest click probability.

[0102] For any first sample business information, the electronic device determines the sample search words that make the click probability corresponding to the first sample business information greater than the preset probability value, and for the sample search words that make the click probability corresponding to the first sample business information greater than the preset probability value, select the third number of sample search words with the highest click probability in order of click probability from large to small or from small to large, and form the third number of sample search words into a search word set.

[0103] The search term set can be expressed as d j Represents the j-th first sample business information.

[0104] (D-3) The electronic device performs word segmentation on the first sample business information and the plurality of sample search words to obtain a first set and a plurality of second sets.

[0105] One sample search term corresponds to one second set.

[0106] For any first sample service information, the electronic device may use a word segmenter to segment the first sample service information to obtain a first set, and the first set may be expressed as: in, represents the first set, represents the kth word in the jth first sample business information, tokenizer(d j ) indicates that d j To perform word segmentation, d j Represents the j-th first sample business information.

[0107] Correspondingly, for any sample search word, the electronic device may also use a word segmenter to segment the sample search word to obtain a second set, which may be expressed as: in, represents the second set, represents the k′th word in the i-th sample search term, tokenizer(q i ) indicates that q i To perform word segmentation, q i represents the i-th sample search term, where k and k′ are both integers greater than 0.

[0108] (D-4) The electronic device determines first weights corresponding to the multiple words included in the field based on the search word set, the first set and the multiple second sets.

[0109] For any first sample business information, the electronic device traverses the sample search terms in the search term set based on the search term set, determines the probability that multiple terms in the first set are terms in the second set corresponding to a third number of sample search terms, determines the second weights corresponding to the multiple terms in the first set based on the probability that the multiple terms in the first set are terms in the second set corresponding to the third number of sample search terms, normalizes the second weights corresponding to the multiple terms in the first set, and obtains the first weights corresponding to the multiple terms.

[0110] In this implementation, for any word in the first set corresponding to the first sample business information, the electronic device can traverse the sample search words in the search word set corresponding to the first sample business information, determine whether the word in the second set corresponding to the sample search word in the search word set is the same as the word in the first set, determine the target sample search word in the second set that is the same as the word in the first set, determine the sum of the click probabilities of the first sample business information under each target sample search word, and obtain the second weight corresponding to the word of the first sample business information.

[0111] For any word in the first set corresponding to the first sample service information, the electronic device can express the second weight corresponding to the word by the following formula:

[0112]

[0113] in, Used to represent the second weight of the kth word in the jth first sample business information, 1(·) represents an indicator function, when the Boolean value entered in the brackets is true, 1(·) is 1, when the Boolean value entered in the brackets is false, 1(·) is 0.

[0114] For the sake of distinction, the target search word selected from the search word set is called the first target search word. According to the formula, it can be seen that for d j The kth word in d j Corresponding search term set For the sample search terms in , for the i-th sample search term, when the k′th term in the i-th sample search term matches d j When the kth words in the search word set are the same, 1(·) is 1, and the electronic device determines that the search word set The sample search terms include multiple words with d j The second target search term that is the same as the kth term in d j The sum of the click probabilities under each second target search term is obtained as d j The second weight corresponding to the kth word in .

[0115] For example, Only q i The k′th word in this sample search term is the same as d j The kth word in d is the same, then j The second weight corresponding to the kth word in is d j In the sample search term q i The corresponding click probability p ij .

[0116] After the electronic device determines the second weights respectively corresponding to the multiple words in the first set, the second weights respectively corresponding to the multiple words in the first set can be normalized by the following formula, which can be expressed as:

[0117]

[0118] in, represents the first weight of the kth word in the jth first sample business information.

[0119] After determining the first weights respectively corresponding to the multiple words of the first sample business information, the electronic device determines the multiple words included in the title field, and obtains the first weights of the multiple words included in the title field.

[0120] (C-2) If the masking method corresponding to the field is a word-level masking method, the field type of the field is a title field type, and based on the seventh probability, a third target word is randomly selected from the field, the electronic device randomly selects the third target word from the field and performs word-level masking on the third target word.

[0121] If it is determined based on the seventh probability that the third target word is randomly selected from the field, the electronic device may randomly select the seventh preset ratio of words from the field, determine whether the number of the seventh preset ratio of words exceeds the sixth preset threshold, and if the number of the seventh preset ratio of words does not exceed the sixth preset threshold, the electronic device uses the seventh preset ratio of words as the third target word for word-level masking. If the number of the seventh preset ratio of words exceeds the sixth preset threshold, the electronic device randomly selects the sixth preset threshold of words from the seventh preset ratio of words, and uses the sixth preset threshold of words as the third target word for word-level masking.

[0122] In another possible implementation, for any field in the first sample business information, if the masking method corresponding to the field is a word-level masking method, and the field type of the field is a keyword field type, the electronic device can randomly select a fourth target word from the field and perform word-level masking on the fourth target word.

[0123] In this implementation, the electronic device may randomly select words of the eighth preset ratio from the field, determine whether the number of words of the eighth preset ratio exceeds the seventh preset threshold, and if the number of words of the eighth preset ratio does not exceed the seventh preset threshold, the electronic device uses the words of the eighth preset ratio as the fourth target word for word-level masking. If the number of words of the eighth preset ratio exceeds the seventh preset threshold, the electronic device randomly selects words of the seventh preset threshold from the words of the eighth preset ratio, and uses the words of the seventh preset threshold as the fourth target word for word-level masking.

[0124] The eighth preset ratio can be set and changed as needed, and is not specifically limited. Here, the eighth preset ratio of 15% is used as an example for explanation.

[0125] If the field type of the field is a keyword field type, it indicates that the field is a keyword field. When the electronic device divides the first sample service information into a plurality of fields, the electronic device may determine the keyword field through the following steps (E-1) to (E-4), including:

[0126] (E-1) The electronic device sorts the multiple words in the first sample business information based on the first weights respectively corresponding to the multiple words in the first sample business information to obtain a candidate keyword sequence.

[0127] The electronic device may sort multiple words in the first sample business information in descending order according to the first weight to obtain a candidate keyword sequence. The electronic device may also sort multiple words in the first sample business information in ascending order according to the first weight to obtain a candidate keyword sequence. There is no specific limitation on this.

[0128] The first weights corresponding to the multiple words in the first sample business information can be implemented through the above steps (D-1) to (D-4), which will not be repeated here.

[0129] (E-2) The electronic device selects a target search word from the search word set corresponding to the first sample business information, and determines a word corresponding to the target search word and words corresponding to other search words in the candidate keyword sequence.

[0130] The remaining search terms are search terms other than the target search term in the search term set.

[0131] The electronic device may randomly select a sample search word from the search word set corresponding to the first sample business information as a target search word, and determine a word corresponding to the target search word and words corresponding to other search words in the candidate keyword sequence.

[0132] (E-3) The electronic device adjusts the position of the word corresponding to the target search word to the front of the candidate keyword sequence, and adjusts the positions of the words corresponding to the remaining search words to the back of the position of the word corresponding to the target search word, to obtain a rearranged candidate keyword sequence.

[0133] The electronic device places the word position corresponding to the target search word at the front of the candidate keyword sequence, and places the word positions corresponding to the remaining search words behind the word position corresponding to the target search word, thereby obtaining a rearranged candidate keyword sequence.

[0134] (E-4) The electronic device selects a fourth number of words from the rearranged candidate keyword sequence to obtain a keyword field.

[0135] The electronic device selects a fourth number of words at the front positions from the rearranged candidate keyword sequence, and forms a keyword field with the fourth number of words.

[0136] It can be seen from this that the keyword field is composed of multiple words, and it is difficult to infer the complete word through the context. Therefore, when the electronic device masks the keyword field, it only performs word-level masking.

[0137] The longer the field is, the longer the model recognition time is and the greater the difficulty of recognition is. Therefore, in an embodiment of the present application, for extra-long fields that exceed the maximum length of the model input sequence, the electronic device determines the keyword field based on word weights. This can compress the extra-long fields and prevent the fields from exceeding the maximum length of the model input sequence, thereby shortening the model recognition time and reducing the difficulty of model recognition.

[0138] In summary, the electronic device adopts an alternating masking method for address fields, brand fields, and other types of fields, and adopts a word-level masking method for category fields, title fields, and keyword fields. Although the category fields, title fields, and keyword fields all adopt a word-level masking method, different fields have different methods for selecting target words. It can be seen that when masking, the embodiment of the present application adopts different masking strategies for different fields, so that the model can fully learn the semantic information of multiple fields, thereby improving the reasoning ability of the model.

[0139] Step 304: the electronic device concatenates the target search term corresponding to the second sample business information to the front of the second sample business information to obtain fourth sample business information.

[0140] The target search word is a search word in a search word set corresponding to the second sample business information, and the search word set is determined based on multiple sample search words.

[0141] The second sample business information is obtained based on the first sample business information. Therefore, the search term set corresponding to the second sample business information is the search term set corresponding to the first sample business information. The target search term is the search term selected from the search term set when determining the keyword field.

[0142] Since the first sample business information is composed of multiple fields, the second sample business information obtained after masking is also composed of multiple fields. The electronic device then splices the target search term to the front of the second sample business information to obtain the fourth sample business information, which is also composed of the target search term and multiple fields.

[0143] In an embodiment of the present application, since the search terms in the search scenario are more colloquial and the business information is more written, in order to narrow the vocabulary difference between the search terms and the business information, the electronic device randomly selects a sample search term from the search term set corresponding to the sample business information and splices it to the front of the sample business information, and then trains the sample search term and the sample business information at the same time, thereby improving the reasoning ability of the model.

[0144] It should be noted that the electronic device may first concatenate the target search word to the front of the first sample business information, and then mask the words in multiple fields of the first sample business information by executing steps 302 to 304 to obtain the fourth sample business information, and this is not specifically limited.

[0145] Step 305: The electronic device determines the actual probability and the predicted probability that the fifth target word is masked.

[0146] The fifth target word is any word in the target field, and the target field is any field in the fourth sample business information.

[0147] The process of the electronic device determining the actual probability of the fifth target word being masked may be: for any word in any field of the fourth sample business information, the electronic device determines the number of words masked by the word at the word level and the total number of words masked by multiple words in the field at the word level, and the electronic device determines the ratio of the number of words masked by the word to the total number to obtain the actual probability of the fifth target word being masked. Alternatively, the electronic device may also determine the total number of words masked by multiple words in the field at the word level, and the electronic device determines the inverse of the total number to obtain the actual probability of the fifth target word being masked. If the word is not masked, the corresponding actual probability is 0.

[0148] The process of the electronic device determining the predicted probability of the fifth target word being masked may be: the electronic device inputs the fourth sample service information into the model, predicts the probability of the fifth target word being masked by the model, and obtains the predicted probability corresponding to the fifth target word.

[0149] In this implementation, the electronic device can first determine the position identifiers corresponding to multiple words in the target search word in the fourth sample business information and the position identifiers corresponding to multiple other words except the target search word, the type identifiers and word identifiers corresponding to multiple fields included in the fourth sample business information and the type identifier and word identifier corresponding to the target search word, input the position identifier, type identifier and word identifier corresponding to the fourth sample business information into the model, and predict the probability of the fifth target word being masked through the model.

[0150] One point that needs to be explained is that when users browse the returned business information, they generally browse from left to right to determine whether the business information is related to the search term. Therefore, the electronic device can start counting from a preset value and determine the position identifiers corresponding to the multiple words in the target search term in order of position. Correspondingly, for the multiple other words after the target search term in the fourth sample business information, the electronic device can also start counting from the preset value and determine the position identifiers corresponding to the multiple other words in order of position.

[0151] Since the words in the target search term are counted from the preset value, the words in the other words are also counted from the preset value, therefore, the position identifiers corresponding to the target search term and the words in the same position in the other words are the same, and the same position identifiers will be encoded to obtain the same position vector. The preset value can be set and changed as needed, and there is no specific limitation on this. For example, the preset value is 0 or 1.

[0152] In addition, different fields correspond to different type identifiers, which makes it easy to distinguish different fields.

[0153] See also Figure 4 ,from Figure 4 It can be seen that the target search word is in front of the second sample business information, and the position identifiers corresponding to the multiple words in the target search word are counted from 0, and the position identifiers corresponding to the multiple other words are also counted from 0. The target search word and multiple fields also correspond to different type identifiers and word identifiers. The electronic device inputs the word identifier, position identifier, and type identifier corresponding to the fourth sample business information into the model, and predicts the probability of the fifth target word in the fourth sample business information being masked through the model.

[0154] Step 306: The electronic device performs model training based on the actual probability and predicted probability corresponding to the fifth target word to obtain a semantic recognition model.

[0155] The electronic device determines the product of the predicted probability and the actual probability corresponding to the fifth target word, adjusts the model parameters based on the change value of the product gradient to reduce the gradient change value, and re-determines the gradient change value of the product of the predicted probability and the actual probability of the fifth target word based on the adjusted model parameters, and then continues to adjust the model parameters until the gradient change value is within a preset change range, or the number of iterations reaches the maximum number of iterations, the model training is completed, and a semantic recognition model is obtained.

[0156] In the embodiment of the present application, the electronic device may represent the fourth sample service information as: Wherein, N is used to represent the number of fields other than the target search term in the fourth sample business information. Indicates D i′ The nth field in represents the j′th word in the nth field, |F n | represents the number of words in the nth field, and n, N, j′ and i′ are all integers greater than 0.

[0157] Based on the representation of the fourth sample business information, the electronic device can define the following objective function, adjust the model parameters by the change value of the objective function gradient, and when the gradient change value of the objective function is within the preset change range, or the number of iterations reaches the maximum number of iterations, the electronic device determines that the model converges and obtains a semantic recognition model. The objective function can be expressed as:

[0158]

[0159] in, Express The number of words masked in Indicates the total number of words masked in the nth field. express The actual probability of express The predicted probability of , θ represents the model parameters.

[0160] In an embodiment of the present application, the initial model can use the Adam optimizer and use stochastic gradient descent to update the model parameters. The number of layers, hidden layer dimensions, number of attention heads, full connection dimensions, and the number of sample business information per batch (batchsize) of the initial model can be set and changed as needed. For example, the number of layers of the model is 24, the hidden layer dimension is 1024, the number of attention heads is 16, the full connection dimension is 4096, and the number of sample business information in a batch is 20.

[0161] An embodiment of the present application provides a method for training a semantic recognition model. When training the semantic recognition model, the method first divides the first sample business information into multiple fields, and then masks the words in the multiple fields respectively according to the field types corresponding to the multiple fields. Since the words in the multiple fields are masked, the model can fully learn the semantic information of the multiple fields, thereby improving the reasoning ability of the semantic recognition model.

[0162] In the embodiment of the present application, the semantic recognition model trained by the above method is a pre-trained semantic recognition model, and the pre-trained semantic recognition model is further trained by labeled sample search terms and sample business information to obtain the final target semantic recognition model. The pre-trained semantic recognition model obtained by the method provided in the embodiment of the present application has a relevance score (Q-AUC) of 1.7 percentage points on the test set of labeled sample search terms and sample business information manually annotated offline.

[0163] In addition, in the embodiment of the present application, by discarding other fields except the title field, the importance of different fields to the relevance task is analyzed, and it is found that each field has a positive effect on the relevance task. Specifically, the performance of the model on the relevance task is compared after the related art only includes sample search terms and title fields, the present application includes sample search terms and multiple fields, and discards other fields except the title field. The experimental results can be seen in Figure 5 .

[0164] from Figure 5 It can be seen that the relevance score in the related art is only 86.52%, while in this application, when sample search terms and multiple fields are included, the relevance score is 88.51%, and in this application, when the keyword field is discarded and other fields are retained, the relevance score is 87.95%, when the address field is discarded and other fields are retained, the relevance score is 88.38%, when the category field is discarded and other fields are retained, the relevance score is 88.41%, and when the brand field is discarded and other fields are retained, the relevance score is 88.42%. It can be seen that according to the method provided by this application, the relevance scores determined are higher than the relevance scores determined by the method in the related art, indicating that the reasoning ability of the semantic recognition model provided by this application has indeed been improved.

[0165] Figure 6 is a structural diagram of a training device for a semantic recognition model provided in an embodiment of the present application, see Figure 6 , the device comprises:

[0166] An acquisition module 601 is used to acquire a sample search term and first sample business information;

[0167] A first determining module 602 is used to determine masking modes corresponding to the multiple fields of the first sample business information based on field types corresponding to the multiple fields of the first sample business information, where the multiple fields are obtained by dividing the first sample business information based on semantic information, and one field corresponds to one masking mode;

[0168] A masking module 603, configured to mask the words in the multiple fields of the first sample business information respectively based on the masking modes respectively corresponding to the multiple fields of the first sample business information, to obtain the second sample business information;

[0169] The training module 604 is used to perform model training based on the sample search terms and the second sample business information to obtain a semantic recognition model.

[0170] In one possible implementation, the masking module 603 is used to, if the masking method of the field is an alternating masking method, and based on the first probability, it is determined that the word-level masking method is currently used for the words in the field, select a first target word from the field, and perform word-level masking on the first target word, the first probability is used to represent the probability of performing word-level masking on the words in the field, and the alternating masking method is used to represent alternating between the word-level masking method and the word-level masking method to mask the words in the field; if the masking method of the field is an alternating masking method, and based on the second probability, it is determined that the word-level masking method is currently used for the words in the field, select a second target word from the field, and perform word-level masking on the second target word, the second probability is used to represent the probability of performing word-level masking on the words in the field.

[0171] In another possible implementation, the masking module 603 is configured to determine a first number of phrases included in the field; select a first preset ratio of phrases from the first number of phrases as first target words, and mask the first target words.

[0172] In another possible implementation, the masking module 603 is used to use the phrases in the first preset proportion as the first target words if the number of phrases in the first preset proportion does not exceed the first preset threshold; and to select the phrases in the first preset threshold from the phrases in the first preset proportion as the first target words if the number of phrases in the first preset proportion exceeds the first preset threshold.

[0173] In another possible implementation, the masking module 603 is used to, if the masking method of the field is a word-level masking method, the field type of the field is a category field type, and the preset ratio of words to be masked in the field is determined to be a second preset ratio based on the third probability, select words of the second preset ratio from the field, and perform word-level masking on the words of the second preset ratio, and the third probability is used to represent the probability of masking the words of the second preset ratio in the field; if the masking method of the field is a word-level masking method, the field type of the field is a category field type, and the preset ratio is determined to be a third preset ratio based on the fourth probability, select the third preset ratio from the field words, perform word level masking on the third preset proportion of words in the field, and the fourth probability is used to represent the probability of masking the third preset proportion of words in the field; if the masking method of the field is the word level masking method, the field type of the field is the category field type, and the preset proportion is determined to be the fourth preset proportion based on the fifth probability, select the fourth preset proportion of words from the field, perform word level masking on the fourth preset proportion of words, and the fifth probability is used to represent the probability of masking the fourth preset proportion of words in the field, the third probability is greater than the fourth probability, the fourth probability is greater than the fifth probability, the fourth preset proportion is greater than the third preset proportion, and the third preset proportion is greater than the second preset proportion.

[0174] In another possible implementation, the masking module 603 is used to, if the masking method of the field is a word-level masking method, the field type of the field is a title field type, and based on the sixth probability, determine to select a third target word from the field according to the size of the word weight, select the third target word from the field based on the first weights corresponding to the multiple words included in the field, and perform word-level masking on the third target word, one word corresponds to one first weight, and the sixth probability is used to represent the probability of selecting the third target word according to the size of the word weight; if the masking method of the field is a word-level masking method, the field type of the field is a title field type, and based on the seventh probability, determine to randomly select a third target word from the field, randomly select the third target word from the field, and perform word-level masking on the third target word, and the seventh probability is used to represent the probability of randomly selecting the third target word.

[0175] In another possible implementation, the number of sample search terms is multiple;

[0176] The device also includes:

[0177] A second determination module is used to determine the click probabilities corresponding to the first sample business information based on the number of clicks corresponding to the first sample business information, where the number of clicks is used to indicate the number of times the first sample business information is clicked when searching for the first sample business information through a sample search term;

[0178] A third determination module is used to determine a search word set corresponding to the first sample business information based on the click probabilities respectively corresponding to the first sample business information, the search word set including a third number of sample search words, and the first sample business information corresponding to the third number of sample search words has the highest click probability;

[0179] A word segmentation module, used to segment the first sample business information and the multiple sample search terms respectively to obtain a first set and multiple second sets, where one sample search term corresponds to one second set;

[0180] The fourth determination module is used to determine first weights corresponding to multiple words included in the field based on the search word set, the first set and multiple second sets.

[0181] In another possible implementation, the fourth determination module is used to traverse the sample search terms in the search term set based on the search term set, and determine the probability that multiple terms in the first set are terms in the second set corresponding to a third number of sample search terms; determine the second weights corresponding to the multiple terms in the first set based on the probability that the multiple terms in the first set are terms in the second set corresponding to the third number of sample search terms; and normalize the second weights corresponding to the multiple terms in the first set to obtain the first weights corresponding to the multiple terms.

[0182] In another possible implementation, the masking module 603 is used to randomly select a fourth target word from the field and perform word-level masking on the fourth target word if the masking mode of the field is word-level masking and the field type of the field is a keyword field type.

[0183] In another possible implementation, the device further includes:

[0184] A sorting module, configured to sort the multiple words in the first sample business information based on the first weights respectively corresponding to the multiple words in the first sample business information to obtain a candidate keyword sequence;

[0185] a fifth determination module, configured to select a target search word from the search word set corresponding to the first sample business information, and determine a word corresponding to the target search word and words corresponding to other search words in the candidate keyword sequence, where the other search words are search words other than the target search word in the search word set;

[0186] An adjustment module, used to adjust the position of the word corresponding to the target search word to the front of the candidate keyword sequence, and adjust the positions of the words corresponding to the remaining search words to the back of the position of the word corresponding to the target search word, so as to obtain a rearranged candidate keyword sequence;

[0187] The selection module is used to select a fourth number of words from the rearranged candidate keyword sequence to obtain a keyword field.

[0188] In another possible implementation, the device further includes:

[0189] a discarding module, configured to discard part of the other fields among the multiple other fields except the title field included in the first sample service information, so as to obtain the third sample service information;

[0190] The masking module 603 is further configured to use the third sample service information as the first sample service information, and determine masking modes corresponding to the multiple fields of the first sample service information based on the field types corresponding to the multiple fields of the first sample service information.

[0191] In another possible implementation, the number of sample search terms is multiple;

[0192] The training module 604 is used to concatenate the target search term corresponding to the second sample business information to the front of the second sample business information to obtain the fourth sample business information, where the target search term is a search term in the search term set corresponding to the second sample business information, and the search term set is determined based on multiple sample search terms; respectively determine the actual probability and predicted probability that the fifth target term is masked, where the fifth target term is any term in the target field, and the target field is any field in the fourth sample business information; perform model training based on the actual probability and predicted probability corresponding to the fifth target term to obtain a semantic recognition model.

[0193] An embodiment of the present application provides a training device for a semantic recognition model. When training the semantic recognition model, the device first divides the first sample business information into multiple fields, and then masks the words in the multiple fields respectively according to the field types corresponding to the multiple fields. Since the words in the multiple fields are masked, the model can fully learn the semantic information of the multiple fields, thereby improving the reasoning ability of the semantic recognition model.

[0194] It should be noted that: the training device for the semantic recognition model provided in the above embodiment only uses the division of the above functional modules as an example when training the semantic recognition model. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the electronic device is divided into different functional modules to complete all or part of the functions described above. In addition, the training device for the semantic recognition model provided in the above embodiment and the training method embodiment of the semantic recognition model belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0195] If the electronic device is provided as a terminal, please refer to Figure 7 , Figure 7 The structure block diagram of a terminal 700 provided by an exemplary embodiment of the present application is shown. The terminal 700 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 700 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0196] Typically, the terminal 700 includes a processor 701 and a memory 702 .

[0197] The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0198] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one program code instruction, which is used to be executed by the processor 701 to implement the training method of the semantic recognition model provided in the method embodiment of the present application.

[0199] In some embodiments, the terminal 700 may further optionally include: a peripheral device interface 703 and at least one peripheral device. The processor 701, the memory 702 and the peripheral device interface 703 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 703 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 704, a touch display screen 705, a camera 706, an audio circuit 707, a positioning component 708 and a power supply 709.

[0200] The peripheral device interface 703 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 701 and the memory 702. In some embodiments, the processor 701, the memory 702, and the peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral device interface 703 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0201] The radio frequency circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 704 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 704 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 704 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0202] The display screen 705 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen 705 is a touch display screen, the display screen 705 also has the ability to collect touch signals on the surface or above the surface of the display screen 705. The touch signal can be input to the processor 701 as a control signal for processing. At this time, the display screen 705 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 705 can be one, and the front panel of the terminal 700 is set; in other embodiments, the display screen 705 can be at least two, which are respectively set on different surfaces of the terminal 700 or are folded; in some other embodiments, the display screen 705 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 700. Even, the display screen 705 can also be set as a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 705 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0203] The camera assembly 706 is used to capture images or videos. Optionally, the camera assembly 706 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0204] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 701 for processing, or input them into the radio frequency circuit 704 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 700. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0205] Positioning component 708 is used to locate the current geographical location of terminal 700 to implement navigation or LBS (Location Based Service). Positioning component 708 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

[0206] The power supply 709 is used to power various components in the terminal 700. The power supply 709 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 709 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0207] In some embodiments, the terminal 700 further includes one or more sensors 710 , including but not limited to: an acceleration sensor 711 , a gyroscope sensor 712 , a pressure sensor 713 , an optical sensor 714 , and a proximity sensor 714 .

[0208] The acceleration sensor 711 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 700. For example, the acceleration sensor 711 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 701 can control the touch screen 705 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 711. The acceleration sensor 711 can also be used for collecting game or user motion data.

[0209] The gyro sensor 712 can detect the body direction and rotation angle of the terminal 700, and the gyro sensor 712 can cooperate with the acceleration sensor 711 to collect the user's 3D actions on the terminal 700. The processor 701 can implement the following functions based on the data collected by the gyro sensor 712: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0210] The pressure sensor 713 can be set on the side frame of the terminal 700 and / or the lower layer of the touch display screen 705. When the pressure sensor 713 is set on the side frame of the terminal 700, it can detect the user's holding signal of the terminal 700, and the processor 701 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 713. When the pressure sensor 713 is set on the lower layer of the touch display screen 705, the processor 701 controls the operability controls on the UI interface according to the user's pressure operation on the touch display screen 705. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0211] The optical sensor 714 is used to collect the ambient light intensity. In one embodiment, the processor 701 can control the display brightness of the touch display screen 705 according to the ambient light intensity collected by the optical sensor 714. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 705 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 705 is reduced. In another embodiment, the processor 701 can also dynamically adjust the shooting parameters of the camera assembly 706 according to the ambient light intensity collected by the optical sensor 714.

[0212] The proximity sensor 715, also called a distance sensor, is usually arranged on the front panel of the terminal 700. The proximity sensor 715 is used to collect the distance between the user and the front of the terminal 700. In one embodiment, when the proximity sensor 715 detects that the distance between the user and the front of the terminal 700 is gradually decreasing, the processor 701 controls the touch display screen 705 to switch from the screen-on state to the screen-off state; when the proximity sensor 715 detects that the distance between the user and the front of the terminal 700 is gradually increasing, the processor 701 controls the touch display screen 705 to switch from the screen-off state to the screen-on state.

[0213] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the terminal 700, and the terminal 700 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0214] If the electronic device is provided as a server, see Figure 8 , Figure 8800 is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server 800 may have relatively large differences due to different configurations or performances, and may include a processor (central processing units, CPU) 801 and a memory 802, wherein the memory 802 stores at least one program code, and the at least one program code is loaded and executed by the processor 801 to implement the methods provided in the above-mentioned various method embodiments. Of course, the server 800 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server 800 may also include other components for implementing device functions, which will not be described in detail here.

[0215] If the electronic device is provided as a terminal and a server, the structural block diagrams of the terminal and the server can be respectively referred to Figure 7 and Figure 8 .

[0216] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the training method of the semantic recognition model in the above embodiment.

[0217] In an exemplary embodiment, a computer program product is also provided. The computer program product stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the training method of the semantic recognition model in the above embodiment.

[0218] In some embodiments, the computer program involved in the embodiments of the present application may be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain system.

[0219] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0220] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a semantic recognition model, characterized in that: The method comprises: Obtaining sample search terms and first sample business information; Determining, based on field types respectively corresponding to multiple fields of the first sample service information, masking modes respectively corresponding to multiple fields of the first sample service information, wherein the multiple fields are obtained by dividing the first sample service information based on semantic information, and one field corresponds to one masking mode; Based on masking modes corresponding to the multiple fields of the first sample business information, respectively masking the words in the multiple fields of the first sample business information to obtain the second sample business information; Based on the sample search terms and the second sample business information, model training is performed to obtain a semantic recognition model; The process of masking the words in the field based on the masking method corresponding to the field includes: If the masking mode of the field is a word-level masking mode, and the field type of the field is a keyword field type, a fourth target word is randomly selected from the field, and word-level masking is performed on the fourth target word.

2. The method according to claim 1, characterized in that The process of masking the words in the field based on the masking method corresponding to the field includes: If the masking mode of the field is an alternating masking mode, and it is determined based on a first probability that a word-level masking mode is currently used for the words in the field, a first target word is selected from the field, and the first target word is masked at the word level, the first probability is used to represent the probability of masking the words in the field at the word level, and the alternating masking mode is used to represent masking the words in the field by alternating between a character-level masking mode and a word-level masking mode; If the masking method of the field is an alternating masking method, and based on the second probability, it is determined that the word level masking method is currently adopted for the words in the field, a second target word is selected from the field, and the second target word is masked at the word level. The second probability is used to represent the probability of masking the words in the field at the word level.

3. The method according to claim 2, characterized in that The selecting a first target word from the field and performing word-level masking on the first target word includes: determining a first number of phrases included in the field; A first preset ratio of phrases is selected from the first number of phrases as the first target words, and the first target words are masked.

4. The method according to claim 3, characterized in that The step of selecting a first preset ratio of phrases from the first number of phrases as the first target words includes: If the number of the phrases in the first preset ratio does not exceed a first preset threshold, taking the phrases in the first preset ratio as the first target words; If the number of the word groups in the first preset ratio exceeds the first preset threshold, the word groups in the first preset threshold are selected from the word groups in the first preset ratio as the first target words.

5. The method according to claim 1, characterized in that The process of masking the words in the field based on the masking method corresponding to the field includes: If the masking mode of the field is a word-level masking mode, the field type of the field is a category field type, and the preset proportion of words in the field to be masked is determined to be a second preset proportion based on the third probability, words of the second preset proportion are selected from the field, and word-level masking is performed on the words of the second preset proportion, and the third probability is used to represent the probability of masking the words of the second preset proportion in the field; If the masking mode of the field is a word-level masking mode, the field type of the field is a category field type, and the preset ratio is determined to be a third preset ratio based on the fourth probability, words of the third preset ratio are selected from the field, and word-level masking is performed on the words of the third preset ratio, and the fourth probability is used to represent the probability of masking the words of the third preset ratio in the field; If the masking method of the field is the word-level masking method, the field type of the field is the category field type, and the preset ratio is determined to be the fourth preset ratio based on the fifth probability, words of the fourth preset ratio are selected from the field, and the words of the fourth preset ratio are masked at the word level, the fifth probability is used to represent the probability of masking the fourth preset ratio of words in the field, the third probability is greater than the fourth probability, the fourth probability is greater than the fifth probability, the fourth preset ratio is greater than the third preset ratio, and the third preset ratio is greater than the second preset ratio.

6. The method according to claim 1, characterized in that The process of masking the words in the field based on the masking method corresponding to the field includes: If the masking mode of the field is a word-level masking mode, the field type of the field is a title field type, and it is determined based on the sixth probability that a third target word is selected from the field according to the size of the word weight, based on the first weights respectively corresponding to a plurality of words included in the field, the third target word is selected from the field, and word-level masking is performed on the third target word, one word corresponds to one first weight, and the sixth probability is used to represent the probability of selecting the third target word according to the size of the word weight; If the masking method of the field is the word-level masking method, the field type of the field is the title field type, and based on the seventh probability, it is determined to randomly select a third target word from the field, randomly select the third target word from the field, and perform word-level masking on the third target word, the seventh probability is used to represent the probability of randomly selecting the third target word.

7. The method according to claim 6, characterized in that The number of the sample search terms is multiple; The process of determining first weights respectively corresponding to the multiple words included in the field includes: Determining click probabilities corresponding to the first sample business information based on the number of clicks corresponding to the first sample business information, where the number of clicks indicates the number of times the first sample business information is clicked when searching for the first sample business information through a sample search term; Determine, based on the click probabilities respectively corresponding to the first sample business information, a search word set corresponding to the first sample business information, the search word set including a third number of sample search words, and the first sample business information corresponding to the third number of sample search words has the highest click probability; Performing word segmentation on the first sample business information and the multiple sample search terms respectively to obtain a first set and multiple second sets, where one sample search term corresponds to one second set; Based on the search word set, the first set and the plurality of second sets, first weights respectively corresponding to the plurality of words included in the field are determined.

8. The method according to claim 7, characterized in that The determining, based on the search word set, the first set, and the plurality of second sets, first weights respectively corresponding to the plurality of words included in the field comprises: Based on the search word set, traverse the sample search words in the search word set to determine the probability that the plurality of words in the first set are words in the second set corresponding to the third number of sample search words; Determine second weights corresponding to the plurality of words in the first set based on probabilities that the plurality of words in the first set are words in the second set corresponding to the third number of sample search words; The second weights respectively corresponding to the multiple words in the first set are normalized to obtain the first weights respectively corresponding to the multiple words.

9. The method according to claim 1, characterized in that: The process of determining a keyword field from the first sample business information includes: sorting the multiple words in the first sample business information based on first weights respectively corresponding to the multiple words in the first sample business information to obtain a candidate keyword sequence; Selecting a target search word from the search word set corresponding to the first sample business information, and determining a word corresponding to the target search word and words corresponding to other search words in the candidate keyword sequence, wherein the other search words are search words in the search word set other than the target search word; The position of the word corresponding to the target search word is adjusted to the front of the candidate keyword sequence, and the positions of the words corresponding to the remaining search words are adjusted to the back of the position of the word corresponding to the target search word, to obtain a rearranged candidate keyword sequence; A fourth number of words are selected from the rearranged candidate keyword sequence to obtain a keyword field.

10. The method according to claim 1, characterized in that The method further comprises: For a plurality of other fields except the title field included in the first sample service information, some of the other fields are discarded to obtain third sample service information; The third sample service information is used as the first sample service information, and the step of determining the masking modes corresponding to the multiple fields of the first sample service information based on the field types corresponding to the multiple fields of the first sample service information is performed.

11. The method according to claim 1, characterized in that: The number of the sample search terms is multiple; The performing model training based on the sample search term and the second sample business information to obtain a semantic recognition model includes: splicing a target search word corresponding to the second sample business information to the front of the second sample business information to obtain fourth sample business information, wherein the target search word is a search word in a search word set corresponding to the second sample business information, and the search word set is determined based on multiple sample search words; respectively determining an actual probability and a predicted probability of a fifth target word being masked, wherein the fifth target word is any word in a target field, and the target field is any field in the fourth sample service information; Based on the actual probability and the predicted probability corresponding to the fifth target word, model training is performed to obtain the semantic recognition model.

12. A training device for a semantic recognition model, characterized in that: The device comprises: An acquisition module, used to acquire a sample search term and first sample business information; A first determining module, configured to determine masking modes corresponding to the multiple fields of the first sample service information based on field types corresponding to the multiple fields of the first sample service information, wherein the multiple fields are obtained by dividing the first sample service information based on semantic information, and one field corresponds to one masking mode; The masking module is used to mask the words in the multiple fields of the first sample business information respectively based on the masking methods respectively corresponding to the multiple fields of the first sample business information to obtain the second sample business information. The process of masking the words in the fields based on the masking methods corresponding to the fields includes: If the masking mode of the field is a word-level masking mode, and the field type of the field is a keyword field type, randomly selecting a fourth target word from the field, and performing word-level masking on the fourth target word; The training module is used to perform model training based on the sample search terms and the second sample business information to obtain a semantic recognition model.

13. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the training method of the semantic recognition model as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the training method of the semantic recognition model as described in any one of claims 1 to 11.

Images