Language processing method and language processing system

Abstract

To provide a language processing method.SOLUTION: A language processing method includes: a step of acquiring an initial dataset containing an initial phrase and an initial meaning label related to the initial phrase; a step of training a first meaning classifier on the initial dataset; a step of generating a plurality of expansion phrases corresponding to the initial phrase through sentence pattern reinforcement; a step of generating a first predicted meaning label related to each expansion phrase from the first meaning classifier and a first confidence measure related to the first predicted meaning label; a step of classifying the plurality of expansion phrases into a plurality of expansion subsets on the basis of a comparison between the first predicted meaning label and the initial meaning label, and on the first confidence measure; and a step of training a second meaning classifier for identifying a meaning of an input phrase in a conversation on the basis of some of the plurality of expansion subsets in a curriculum learning form.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a language processing method and a language processing system, and more particularly to a semantic classification technique in a language processing method and a language processing system. [Background technology]

[0002] A large language model (LLM) is an artificial intelligence model that understands the meaning of input characters and generates response character content similar to human responses. LLMs process and generate characters using deep learning techniques, such as architectures like transformer models. To process input characters and generate reasonable interactive responses, LLMs must distinguish the meaning behind the input characters and generate meaningful responses. do It is necessary. Summary of the Invention [Means for solving the problem]

[0003] One aspect of the present disclosure is a language processing method comprising: obtaining an initial dataset including a plurality of initial phrases and a plurality of initial semantic labels associated with the initial phrases; training a first semantic classifier based on the initial dataset; performing sentence pattern expansion corresponding to the initial phrases to generate a plurality of expanded phrases; generating from the first semantic classifier a plurality of first predicted semantic labels associated with the expanded phrases and a plurality of first confidence measures for the first predicted semantic labels; classifying the expanded phrases into a plurality of expanded subsets based on a comparison result between the initial semantic labels and the first predicted semantic labels and the first confidence measure; and training a second semantic classifier to identify the meaning of input phrases in a conversation based on a portion of the expanded subsets in a curriculum learning format.

[0004] Another aspect of the present disclosure is a language processing system including: a storage unit for storing a plurality of computer-executable instructions; and a processing unit coupled to the storage unit that executes the computer-executable instructions to perform the following: obtaining an initial dataset including a plurality of initial phrases and a plurality of initial semantic labels associated with the initial phrases; training a first semantic classifier based on the initial dataset; performing sentence pattern expansion corresponding to the initial phrases to generate a plurality of expanded phrases; generating a plurality of first predicted semantic labels associated with the expanded phrases and a plurality of first confidence measures for the first predicted semantic labels by the first semantic classifier; classifying the expanded phrases into a plurality of expanded subsets based on a comparison result between the initial semantic labels and the first predicted semantic labels and the first confidence measure; and training a second semantic classifier to identify the meaning of input phrases in a conversation based on a portion of the expanded subsets in a curriculum learning manner.

[0005] In some embodiments, the language processing method and system can perform semi-supervised filtering on the generated expanded phrases, where the semi-supervised filtering is used to select appropriate phrases from the expanded phrases and input them into a training agent program to train the semantic classifier, and the above data expansion and semi-supervised filtering can achieve end-to-end automated execution. The second and third semantic classifiers trained in a curriculum learning manner can achieve good generality / versatility (because they include more types of expanded phrases) and maintain high accuracy.

[0006] It should be understood that the foregoing description and the following detailed description are used to exemplify the present application and to aid in the interpretation and understanding of the general principles of the invention claimed herein. [Brief explanation of the drawings]

[0007] To make the above and other objects, features and embodiments of the present disclosure more clearly comprehensible, the description of the drawings is as follows. [Figure 1] FIG. 1 is a schematic diagram of a language processing system in accordance with some embodiments of the present disclosure. [Figure 2] 1 is a flowchart of a language processing method in some embodiments of the present disclosure. [Figure 3] FIG. 1 is a schematic diagram of an initial data set and an expanded data set in some embodiments of the present disclosure. [Figure 4] FIG. 1 is a schematic diagram of an extended data set and a first predicted data set in some embodiments of the present disclosure. [Figure 5] FIG. 10 is a schematic diagram of a plurality of expanded subsets formed after classifying a first prediction dataset in some embodiments of the present disclosure. [Figure 6] FIG. 10 is a schematic diagram of training a second semantic classifier based on a portion of the expanded subset after classification in a process according to some embodiments of the present disclosure. [Figure 7] FIG. 1 is a schematic diagram of an extended data set and a second predicted data set in some embodiments of the present disclosure. [Figure 8] FIG. 10 is a schematic diagram of a plurality of updated expanded subsets formed after classifying a second prediction dataset in some embodiments of the present disclosure. [Figure 9] FIG. 10 is a schematic diagram of training a third semantic classifier based on a portion of the updated expanded subset after classification in some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0008] The following disclosure provides many different embodiments or examples for implementing different features of the present disclosure. Components and configurations in specific examples are used in the following discussion to simplify the disclosure. Any examples discussed are for illustrative purposes only and do not limit the scope and meaning of the disclosure or its examples. Where appropriate, the same reference numerals are used in the drawings and corresponding text descriptions to represent the same or similar components.

[0009] Please refer to Figure 1, which is a schematic diagram of a language processing system 100 in some embodiments of the present disclosure. As shown in Figure 1, the language processing system 100 includes a storage unit 120, a processing unit 140, and a user interface 160. In some embodiments, the language processing system 100 may be a computer, a smartphone, a tablet computer, a video processing server, a data processing server, a server, a tensor computing server, or any equivalent processing device.

[0010] As shown in FIG. 1, the storage unit 120 stores an initial data set Dini including a plurality of initial phrases Pini, initial semantic labels Tini associated with the initial phrases, and a plurality of initial answers Aini to the initial phrases Pini.

[0011] In some embodiments, the initial dataset Dini may be collected from a large language model (LLM) or a question-and-answer (Q&A) dataset stored in a language processing application program.

[0012] 1, the processing unit 140 is used to run a phrase rewriter 142, a training agent program 144, and a conversation engine 146. The conversation engine 146 is capable of running a large-scale language model and is used to process a conversation between the language processing system 100 and a user U1. In some embodiments, the training agent program 144 can train a semantic classifier ICM using initial phrases Pini and initial semantic labels Tini in the initial data set Dini.

[0013] In practical applications, the semantic classifier ICM is a key component of the conversation engine 146. The semantic classifier ICM is used to assist the conversation engine 146 in understanding the purpose or goal behind various input phrases. The semantic classifier ICM is used to distinguish various input phrases into different semantic categories, such as posing a question, making a statement, expressing a command, etc. The semantic classifier ICM can assist the conversation engine 146 in better understanding the meaning of the user's input and more accurately generate appropriate responses to the user's input.

[0014] For example, when a user U1 inputs a phrase P DIA When you enter this phrase, DIA to the conversation engine 146. The conversation engine 146 uses the semantic classifier ICM to classify the input phrase P DIA The meaning corresponding to T DIA This allows the conversation engine 146 to distinguish between the determined meaning T DIA Based on the correct answer A DIA can be generated via the user interface 160. DIA is fed back to the user U1, thereby realizing interactive communication between the language processing system 100 and the user U1.

[0015] In order for the semantic classifier ICM to achieve a high accuracy rate, training the semantic classifier ICM requires a large number of initial phrases P and corresponding initial semantic labels T. In some embodiments, the initial phrases P and corresponding initial semantic labels T may be manually input by an engineer or a user. However, collecting and constructing a large number of initial phrases P and corresponding initial semantic labels T is a difficult task.

[0016] 1 , the processing unit 140 includes a phrase rewriter 142 that can generate a plurality of expanded phrases Paug corresponding to a plurality of initial phrases Pini through sentence augmentation. In some embodiments, the language processing system 100 operates the phrase rewriter 142 to generate a plurality of expanded phrases Paug by sentence augmenting the initial phrases Pini, and performs semi-supervised filtering on the generated expanded phrases Paug, where the semi-supervised filtering is used to select appropriate phrases from the expanded phrases Paug and input them into the training agent program 144 to train the semantic classifier ICM. In some embodiments, the data augmentation and semi-supervised filtering can be performed end-to-end automatically without requiring any additional manual input.

[0017] In some embodiments, phrase rewriter 142, training agent program 144, and conversation engine 146 may be implemented by processing unit 140 executing computer-executable instructions and / or software programs. In some embodiments, processing unit 140 may be a processor, a graphics processor, an application specific integrated circuit (ASIC), or any equivalent processing circuit.

[0018] Please refer to Figure 2, which is a flowchart of a language processing method 200 according to some embodiments of the present disclosure. The language processing method 200 may be performed by the language processing system 100 shown in Figure 1.

[0019] The storage unit 120 is further used to store computer-executable instructions. The processing unit 140 is coupled to the user interface 160 and the storage unit 120, respectively. The processing unit 140 is used to execute the computer-executable instructions and to implement the language processing method 200 described in the following embodiments. The storage unit 120 may be a memory, a hard disk, a fast-access memory, a flash memory, and / or any equivalent data storage element.

[0020] 1 and 2, step S210 is performed to obtain (or receive) an initial data set Dini from a data source (not shown). The initial data set Dini may be stored in the storage unit 120 shown in Fig. 1. In some embodiments, the data source may be a question and answer (Q&A) data server.

[0021] Please refer to FIG. 3, which is a schematic diagram of the initial dataset Dini and the expanded dataset Daug in some embodiments of the present disclosure. As shown in FIG. 3, the initial phrase Pini includes a first initial phrase P1 "I would like to make a reservation at this restaurant," and the initial semantic label Tini includes a first semantic label T1 "Restaurant Reservation" corresponding to the first initial phrase P1. The initial phrase Pini further includes a second initial phrase P2 "I would like to reserve tickets," and the initial semantic label Tini includes a second semantic label T2 "Ticket Reservation" corresponding to the second initial phrase P2. The initial phrase Pini includes a third initial phrase P3 "What's the weather like today?", and the initial semantic label Tini includes a third semantic label T3 "Weather Enquiry" corresponding to the third initial phrase P3.

[0022] As shown in FIGS. 1, 2 and 3, in step S220, the training agent program 144 trains a first semantic classifier ICM1 ​​based on the initial data set Dini.

[0023] In some embodiments, training of the first semantic classifier ICM1 ​​may be based on the following cross-entropy loss function:

[0024]

number

[0025] It should be noted that the first semantic classifier ICM1 ​​is trained based on the initial dataset Dini and includes a limited number of initial phrases Pini and initial semantic labels Tini in the initial dataset Dini. In some embodiments, the number of initial phrases Pini corresponding to each of various initial semantic types included in the initial dataset Dini is small, i.e., there is not enough variation in phrase examples to express similar meanings. In this case, if a user submits a question in which the conversational phrases entered by the user are similar but use different phrasal forms, the first semantic classifier ICM1 ​​(trained based on the initial dataset Dini) may not be able to identify the correct meaning. In other words, the generality / versatility of the initial dataset Dini and the first semantic classifier ICM1 ​​is lacking.

[0026] In order to improve the generality / versatility of the initial data set Dini, step S230 is performed, in which the phrase rewriting device 142 generates a plurality of extended phrases corresponding to the initial phrase Pini in the form of sentence pattern extension. As shown in Fig. 3, a first initial phrase P1 "I want to make a reservation for this restaurant" in the initial phrase Pini can be rewritten by the phrase rewriting device 142 into a plurality of extended phrases P1a1, P1a2, P1a3, P1a4, and P1a5 corresponding to the first initial phrase P1, and all of the extended phrases correspond to the first initial phrase P1.

[0027] In one embodiment, in step S230, a large-scale language model (LLM) can be used to rewrite the initial phrase Pini to generate a large number of expanded phrases Paug. For example, in step S230, a suggestion command, such as "Please rewrite this sentence 'I would like to make a reservation at this restaurant' in a different form," can be input to a large-scale language model (e.g., ChatGPT, Gemini, LLaMA, Mistral AI, Bard, or Copilot), and the response results can be collected from the large-scale language model to generate multiple expanded phrases P1a1 to P1a5.

[0028] In another embodiment, in step S230, a translation model can be used to translate a first initial phrase P1 expressed in a first language (e.g., English) into an intermediate phrase expressed in a second language (e.g., French), and a translation model can be used to translate the intermediate phrase expressed in the second language (e.g., French) back into the first language (e.g., English), and further to generate extended phrases P1a1 to P1a5 expressed in the first language.

[0029] In another embodiment, in step S230, the expanded phrases P1a1 to P1a5 may be generated by replacing a word in the first initial phrase P1 with a synonym related to the word. For example, the word "restaurant" in the first initial phrase P1 may be replaced with a synonym such as "restaurant," "bar," or "dining hall."

[0030] In another embodiment, in step S230, the extended phrases P1a1 to P1a5 may be generated by inserting random noise into the first initial phrase P1. The random noise may be generated by randomly deleting one single character in the first initial phrase P1, randomly changing the order of the single characters in the first initial phrase P1, or randomly adding an additional single character to the first initial phrase P1. The random noise may be used to simulate a user's typing error.

[0031] As shown in Figure 3, the extended data set Daug includes multiple extended phrases P1a1 to P1a5 generated by sentence pattern extension in step S230 based on the first initial phrase P1, and the extended phrases P1a1 to P1a5 correspond to the first semantic label T1 (matching the first initial phrase P1) among the multiple initial semantic labels Tini.

[0032] 2 and 3, in step S230, similarly, a plurality of other expanded phrases P2a1-P2a5 can be generated based on the second initial phrase P2, and a plurality of other expanded phrases P3a1-P3a5 can be generated based on the third initial phrase P3. In the expanded data set Daug, the expanded phrases P2a1-P2a5 correspond to the second semantic label T2 (matching with the second initial phrase P2) among the plurality of initial semantic labels Tini. In the expanded data set Daug, the expanded phrases P3a1-P3a5 correspond to the third semantic label T3 (matching with the third initial phrase P3) among the plurality of initial semantic labels Tini.

[0033] For simplicity, the following paragraphs will be discussed using the expanded phrases P1a1-P1a5 as an example. However, the present disclosure is not limited thereto. Similar operations may be performed on other initial phrases (e.g., the second initial phrase P2 and the third initial phrase P3) to further process the other expanded phrases P2a1-P2a5 and P3a1-P3a5.

[0034] It should be noted that in some embodiments, the expanded phrases P1a1-P1a5 are automatically generated by sentence pattern expansion based on the first initial phrase P1 without artificial supervision. In this case, it is difficult to ensure that all expanded phrases P1a1-P1a5 can maintain their original semantic label T1 ("restaurant reservation"). Typically, most expanded phrases P1a1-P1a5 have the same meaning as the original semantic label T1. However, after sentence pattern expansion, some expanded phrases P1a1-P1a5 may change their meaning or significance. The first semantic label T1 may no longer be suitable for expressing the true intent of some expanded phrases P1a1-P1a5.

[0035] Please refer to Figure 4, which is a schematic diagram of the extended data set Daug and the first predicted data set Daug_P1 associated with step S240 in some embodiments of the present disclosure.

[0036] As shown in Figures 1, 2 and 4, by performing step S240, the processing unit 140 operates the first semantic classifier ICM1 ​​to generate a plurality of first predicted semantic labels TP1 for the extended data set Daug, and generates a plurality of first confidence measures CL1 corresponding to the first predicted semantic labels TP1.

[0037] As in the first prediction data set Daug_P1 shown in FIG. 4, the multiple semantic labels generated by the first semantic classifier ICM1 ​​for the extended phrases P1a1 to P1a5 are T1, T1, T1, T2, and T2, respectively.

[0038] For the plurality of first predicted semantic labels TP1 generated by the first semantic classifier ICM1, the first semantic classifier ICM1 ​​predicts that three expanded phrases P1a1, P1a2, and P1a3 have a first semantic label T1 that is the same as the original semantic label, and predicts that the other two expanded phrases P1a4 and P1a5 have a second semantic label T2 that is different from the original semantic label (i.e., the first semantic label T1). Note that the plurality of first confidence measures CL1 generated by the first semantic classifier ICM1 ​​for the plurality of first predicted semantic labels TP1 are shown in FIG.

[0039] Please refer to Figure 5, which shows a schematic diagram of a plurality of extended subsets DG1 to DG4 formed after classifying the first prediction dataset Daug_P1 in step S250 in some embodiments of the present disclosure.

[0040] As shown in Figures 1, 2 and 5, the processing unit 140 executes step S250 to classify multiple extended phrases P1a1 to P1a5 in the first predicted data set Daug_P1 into multiple extended subsets DG1 to DG4 based on the comparison results between multiple first predicted semantic labels TP1 and multiple initial semantic labels Tini and multiple first confidence criteria CL1.

[0041] In some embodiments, as shown in FIGS. 2 and 5 , the expanded phrase P1a1 is classified into a first expanded subset DG1 because a comparison of the first predicted semantic label TP1 (TP1 of the expanded phrase P1a1 = the first semantic label T1) of the expanded phrase P1a1 with its own initial semantic label T2 shows a match (T2 of the expanded phrase P1a1 = the first semantic label T1), and the first confidence criterion CL1 of the expanded phrase P1a1 (CL1 of the expanded phrase P1a1 is 95%) is greater than a first confidence threshold (which may be set to 80%, for example). The expanded phrase P1a1 in the first expanded subset DG1 has the same predicted semantic label as its initial semantic label, and the first semantic classifier ICM1 ​​has a high confidence criterion for the first expanded subset DG1. In this case, the first expanded subset DG1 is suitable for being added with the highest priority to the training data for training the semantic classifier ICM.

[0042] 2 and 5, the expanded phrases P1a2 and P1a3 are classified into a second expanded subset DG2 because the first predicted semantic labels TP1 (TP1 of the expanded phrases P1a1 and P1a3 = first semantic label T1) of the expanded phrases P1a2 and P1a3 are compared with their initial semantic labels T1 and find a match (T1 of the expanded phrases P1a1 and P1a3 = first semantic label T1), and the first confidence criterion CL1 of the expanded phrases P1a2 and P1a3 (CL1 of the expanded phrases P1a2 and P1a3 is 62% and 50%, respectively) is lower than a first confidence threshold (e.g., 80%). The expanded phrases P1a2 and P1a3 in the second expanded subset DG2 have the same predicted semantic labels as their initial semantic labels, and the first semantic classifier ICM1 ​​has a low confidence criterion for the second expanded subset DG2. In this case, the second expanded subset DG2 is suitable for addition with second priority to the training data for training the semantic classifier ICM.

[0043] In some embodiments, as shown in FIGS. 2 and 5, the expanded phrase P1a4 is classified into the third expanded subset DG3 because the first predicted semantic label TP1 (TP1 of the expanded phrase P1a4 = second semantic label T2) of the expanded phrase P1a4 is compared with its own initial semantic label Tini and the comparison result shows a discrepancy (Tini of the expanded phrase P1a4 = first semantic label T1). The first confidence criterion CL1 of the expanded phrase P1a4 (CL1 of the expanded phrase P1a4 is 82%) is higher than the second confidence threshold (e.g., may be set to 80%). The expanded phrase P1a4 in the third expanded subset DG3 has a predicted semantic label that differs from the initial semantic label, and the first semantic classifier ICM1 ​​has a high confidence criterion for the third expanded subset DG3. In this case, the third expanded subset DG3 is suitable for addition to the training data for training the semantic classifier ICM with a third priority. This may indicate that the meaning of the sentence-expanded expanded phrase P1a4 has changed.

[0044] In some embodiments, as shown in FIGS. 2 and 5 , the expanded phrase P1a5 is classified into the fourth expanded subset DG4 because the comparison of the first predicted semantic label TP1 (TP1 of the expanded phrase P1a5 = the second semantic label T2) and its initial semantic label T2 shows a discrepancy (T2 of the expanded phrase P1a5 = the first semantic label T1) and the first confidence criterion CL1 of the expanded phrase P1a5 (CL1 of the expanded phrase P1a5 is 40%) is lower than the second confidence threshold (which may be set to, for example, 80%). If the expanded phrase P1a5 in the fourth expanded subset DG4 has a predicted semantic label that differs from the initial semantic label and the first semantic classifier ICM1 ​​has a low confidence criterion for the fourth expanded subset DG4, this may indicate difficulty in determining the true meaning of the expanded phrase P1a5 in the fourth expanded subset DG4. For example, the expanded phrase P1a5 after sentence pattern expansion may have an unclear meaning or may be a meaningless sentence. In this case, the fourth expanded subset DG4 is not suitable for training the semantic classifier ICM.

[0045] It should be noted that the first confidence threshold (e.g., 80%) and the second confidence threshold (e.g., 80%) described in the above embodiment are merely illustrative examples. The first confidence threshold and the second confidence threshold are not limited to these specific values. In some other embodiments, more confidence threshold settings can be introduced to further expand the expanded phrases with different confidence criteria. Sub For example, labels with the same meaning are classified into five confidence standards, such as 100% to 81%, 80% to 61%, 60% to 41%, 40% to 21%, and 20% to 0%, and labels with different meanings are classified into five confidence standards, such as 100% to 81%, 80% to 61%, 60% to 41%, 40% to 21%, and 20% to 0%.

[0046] Please refer to Figure 6, which is a schematic diagram of training a second semantic classifier ICM2 based on a portion of the expanded subsets DG1-DG4 after classification in step S260 in some embodiments of the present disclosure.

[0047] As shown in FIGS. 1, 2 and 6, step S260 is executed, and the training agent program 144 trains the second semantic classifier ICM2 based on a portion of the expanded subsets DG1-DG4 in a curriculum learning fashion.

[0048] In some embodiments, the predicted labels that form the basis for classifying the third and fourth expanded subsets DG3 and DG4 are generated based on the first semantic classifier ICM1 ​​established in the initial stage. Because the prediction reliability and confidence generated by the first semantic classifier ICM1 ​​at this time may still be insufficient, the second semantic classifier ICM2 is not trained using the third and fourth expanded subsets DG3 and DG4 during step S260 shown in FIG. 6 .

[0049] 6, in the first round R1 of the curriculum learning mode in step S260, the training agent program 144 trains the second semantic classifier ICM2 based on the initial data set Dini and the first expanded subset DG1. That is, training data with low difficulty and high reliability is preferentially used to train the second semantic classifier ICM2 in the first round R1 of the curriculum learning mode.

[0050] 6, in the second round R2 of the curriculum learning mode in step S260, the training agent program 144 trains the second semantic classifier ICM2 based on the initial data set Dini, the first expanded subset DG1, and the second expanded subset DG2. That is, in the second round R2 of the curriculum learning mode, the training data for training the second semantic classifier ICM2 includes more phrase content and more different variations. In this case, the second semantic classifier ICM2 obtained by training in the curriculum learning mode can have good generality / versatility (because it includes more types of expanded phrases) and can maintain a high standard.

[0051] In some embodiments, training of the second semantic classifier ICM2 may be based on the following cross-entropy loss function:

[0052]

number

[0053] It should be noted that the first semantic classifier ICM1 ​​is trained based on the initial dataset Dini and includes a limited number of initial phrases Pini and initial semantic labels Tini in the initial dataset Dini. In some embodiments, the initial dataset Dini includes a small number of initial phrases Pini corresponding to each of various initial semantic types, i.e., there is not enough variation in phrase examples to express similar meanings. In this case, if the conversational phrases input by the user pose similar questions using different phrasal forms, the first semantic classifier ICM1 ​​(trained based on the initial dataset Dini) may not be able to identify the correct meaning. In other words, the generality / versatility of the initial dataset Dini and the first semantic classifier ICM1 ​​is insufficient. Furthermore, the second semantic classifier ICM2 is trained based on the initial dataset Dini, the first expanded subset DG1, and the second expanded subset DG2. The second semantic classifier ICM2 can achieve better generality / versatility than the first semantic classifier ICM1.

[0054] As shown in FIGS. 1 and 2, in step S310, the second semantic classifier ICM2 is used to classify the input phrase P DIA Meaning of T DIA In step S320, the conversation engine 146 identifies the input phrase P DIA Meaning of T DIA Based on (2nd meaning Classification results based on classifier ICM2) Answer A DIA In some embodiments, the present disclosure is not limited to stopping after training of the second semantic classifier ICM2. More curriculum learning cycles can be performed to further update the semantic classifier ICM, thereby further increasing the accuracy rate of the semantic classifier ICM.

[0055] Please refer to Figure 7, which is a schematic diagram of the extended data set Daug and the second predicted data set Daug_P2 associated with step S270 in some embodiments of the present disclosure.

[0056] As shown in Figures 1, 2 and 7, by performing step S270, the processing unit 140 operates the second semantic classifier ICM2 to generate a plurality of second predicted semantic labels TP2 for the extended data set Daug, and generates a plurality of second confidence measures CL2 corresponding to the second predicted semantic labels TP2.

[0057] As in the first prediction data set Daug_P1 shown in FIG. 4, the multiple semantic labels generated by the first semantic classifier ICM1 ​​for the extended phrases P1a1 to P1a5 are T1, T1, T1, T2, and T2, respectively.

[0058] For the plurality of first predicted semantic labels TP1 generated by the first semantic classifier ICM1, the first semantic classifier ICM1 ​​predicts that three expanded phrases P1a1, P1a2, and P1a3 have the same first semantic label T1 as the original semantic label, and predicts that the other two expanded phrases P1a4 and P1a5 have a second semantic label T2 that is different from the original semantic label (i.e., the first semantic label T1). Note that the plurality of first confidence measures CL1 generated by the first semantic classifier ICM1 ​​for the plurality of first predicted semantic labels TP1 are shown in FIG. In this case, since the second semantic classifier ICM2 has already been trained based on the initial data set Dini and a portion of the extended data set (the first extended subset DG1 and the second extended subset DG2), the second predicted semantic label TP2 and the second confidence criterion CL2 (the second predicted data set Daug_P2 as shown in Figure 7) generated by the second semantic classifier ICM2 may be different from the first predicted semantic label TP1 and the first confidence criterion CL1 predicted by the first semantic classifier ICM1 ​​previously, and as a result of comparison, the second predicted semantic label TP2 and the second confidence criterion CL2 generated by the second semantic classifier ICM2 have a higher standard.

[0059] As shown in FIG. 7, the second semantic classifier ICM2 generates multiple semantic labels for the extended phrases P1a1 to P1a5, which are T1, T1, T1, T2, and T2, respectively, for the second predicted semantic data set Daug_P2. As shown in FIG. 7, the second semantic classifier ICM2 can generate multiple second confidence measures CL2 associated with the multiple second predicted semantic labels TP2. The operation of step S270 is similar to that of step S240. The main difference from step S240 is that step S270 is performed based on the second semantic classifier ICM2 (trained based on the initial data set Dini, the first extended subset DG1, and the second extended subset DG2) rather than the first semantic classifier ICM1 ​​(trained based on the initial data set Dini).

[0060] Please refer to Figure 8, which is a schematic diagram of a plurality of updated extended subsets DG1u to DG4u formed after classifying the second prediction dataset Daug_P2 in step S280 in some embodiments of the present disclosure.

[0061] As shown in Figures 1, 2 and 8, the processing unit 140 executes step S280 to classify multiple extended phrases P1a1 to P1a5 in the second predicted data set Daug_P2 into multiple updated extended subsets DG1u to DG4u based on the comparison results between the multiple second predicted semantic labels TP2 and the multiple initial semantic labels Tini and the multiple second confidence criteria CL2.

[0062] In some embodiments, expanded phrases P1a1 and P1a2 are classified into a first updated expanded subset DG1u, expanded phrase P1a3 into a second updated expanded subset DG2u, expanded phrase P1a4 into a third updated expanded subset DG3u, and expanded phrase P1a5 into a fourth updated expanded subset DG4u, as shown in Figures 2 and 8. The operation of step S280 is similar to that of step S250.

[0063] Please refer to Figure 9, which is a schematic diagram of training a third semantic classifier ICM3 based on a portion of the updated expanded subsets DG1u-DG4u after classification in step S290 in some embodiments of the present disclosure.

[0064] As shown in FIGS. 1, 2 and 9, step S290 is executed, and the training agent program 144 trains the third semantic classifier ICM3 based on a portion of the updated expanded subsets DG1u-DG4u in a curriculum learning fashion.

[0065] In some embodiments, in step S290, the fourth updated expanded subset DG4u is not used to train the third semantic classifier ICM3.

[0066] 9, in the first round R1 of the curriculum learning mode in step S290, the training agent program 144 trains the third semantic classifier ICM3 based on the initial data set Dini and the first updated expanded subset DG1u. That is, training data with low difficulty and high reliability is preferentially used to train the third semantic classifier ICM3 in the first round R1 of the curriculum learning mode.

[0067] 9, in the second round R2 of the curriculum learning mode in step S290, the training agent program 144 trains the third semantic classifier ICM3 based on the initial data set Dini, the first updated expanded subset DG1u, and the second updated expanded subset DG2u. That is, in the second round R2 of the curriculum learning mode, the training data for training the third semantic classifier ICM3 includes more phrase content and more different variations.

[0068] 9, in the third round R3 of the curriculum learning mode in step S290, the training agent program 144 trains a third semantic classifier ICM3 based on the initial data set Dini, the first updated expanded subset DG1, the second updated expanded subset DG2, and the third updated expanded subset DG3. It should be noted that the third updated expanded subset ICM3 generated by the second semantic classifier ICM2 Expansion A second predicted semantic label TP2 associated with the expanded phrase P1a4 in the subset DG3u is generated as ground truth for training this third semantic classifier ICM3, i.e., the expanded phrase P1a4 is no longer considered a phrase associated with the first semantic label T1, but is instead considered a phrase associated with the second semantic label T2 (based on the prediction result of the second semantic classifier ICM2).

[0069] In some embodiments, training of the third semantic classifier ICM3 may be based on the following cross-entropy loss function:

[0070]

number

[0071] As shown in FIGS. 1 and 2, in step S310, the third semantic classifier ICM3 is used to classify the input phrase P input by the user U1. DIA Meaning of T DIA In step S320, the conversation engine 146 identifies the input phrase P DIA Meaning of T DIA Answer A based on the results of classification based on the third semantic classifier ICM3 DIA In some embodiments, the present disclosure is not limited to stopping after training the third semantic classifier ICM3. More curriculum learning cycles can be performed to further update the semantic classifier ICM, thereby further increasing the accuracy rate of the semantic classifier ICM.

[0072] It should be noted that the third semantic classifier ICM3 is trained, and the third updated expanded subset DG3u is used as the training data for the third round R3 of the curriculum learning model. This is because, compared to the third expanded subset DG3 generated in the initial stage based on the first semantic classifier ICM1, the third updated expanded subset DG3u is generated in a subsequent stage based on the prediction results of the second semantic classifier ICM2. The comparison results show that the third updated expanded subset DG3u is more reliable. Therefore, the third updated expanded subset DG3u can be added to the training data to further increase the variety and richness of the expanded data.

[0073] Although specific embodiments of the present disclosure have been disclosed in the above embodiments, the above embodiments are not intended to limit the present disclosure. Various substitutions and improvements can be made in the present disclosure by those skilled in the relevant art without departing from the principle and spirit of the present disclosure. Therefore, the scope of protection of the present disclosure is determined by the scope of the attached patent application. [Explanation of symbols]

[0074] 100: Language Processing System 120: Memory unit 140: Processing unit 142: Phrase rewriting device 144: Training Agent Program 146: Conversation Engine 160: User Interface 200: Language processing methods Aini:Initial answer A DIA :answer CL1: First reliability standard CL2: Second confidence standard Daug: Extended Dataset Daug_P1: First prediction dataset Daug_P2: Second prediction dataset DG1: First extended subset DG2: Second extended subset DG3: Third Extended Subset DG4: Fourth Extended Subset DG1u: First Update Extended Subset DG2u: Second Update Extended Subset DG3u: Third Update Extended Subset DG4u: 4th Update Extended Subset Dini: Initial Dataset ICM: Semantic classifier ICM1: First semantic classifier ICM2: Second semantic classifier ICM3: Third semantic classifier P1: First initial phrase P1a1, P1a2, P1a3, P1a4, P1a5: Extended phrases P2: Second initial phrase P2a1, P2a5: Extended phrases P3: Third initial phrase P3a1, P3a5: Extended phrases Paug: Extended Phrase P DIA :input phrase Pini: Initial phrase R1: First round R2: Second round R3: Third round S210, S220, S230, S240, S250: Process S260, S270, S280, S290: Process S310, S320: Process T1: First semantic label T2: Second semantic label T3: Third semantic label TP1: First predicted semantic label TP2: Second predicted semantic label T DIA :meaning Tini: Initial Semantic Labels U1: User

Claims

1. obtaining an initial data set comprising a plurality of initial phrases and a plurality of initial semantic labels associated with the initial phrases; training a first semantic classifier based on the initial data set; performing a sentence pattern expansion corresponding to the initial phrase to generate a plurality of expanded phrases; generating a plurality of first predicted semantic labels associated with the expanded phrase from the first semantic classifier and a plurality of first confidence measures for the first predicted semantic labels; classifying the expanded phrases into a plurality of expanded subsets based on a comparison between the initial semantic labels and the first predicted semantic labels and the first confidence metric; training a second semantic classifier based in part on the expanded subset in a curriculum learning fashion to identify the meaning of input phrases in conversation; A language processing method comprising: A language processing method, wherein the steps are performed by a language processing system including a processing unit and a storage unit.

2. The step of classifying the expanded phrases into the expanded subsets comprises: classifying a first portion of the expanded subset into a first expanded subset, the first portion having the first predicted semantic label being consistent with the initial semantic label, and the first portion having the first confidence measure being greater than a first confidence threshold; classifying a second portion of the expanded subset into a second expanded subset, the second portion having the first predicted semantic label and the initial semantic label being consistent with each other, and the second portion having the first confidence measure being lower than the first confidence threshold; classifying a third portion of the expanded subset into a third expanded subset, wherein a comparison of the first predicted semantic label and the initial semantic label of the third portion indicates a mismatch, and the first confidence measure of the third portion is higher than a second confidence threshold; classifying a fourth portion of the expanded subset into a fourth expanded subset, wherein a comparison result between the first predicted semantic label and the initial semantic label of the fourth portion shows a mismatch and the first confidence measure of the fourth portion is lower than the second confidence threshold; The language processing method according to claim 1 , comprising:

3. training the second semantic classifier in a curriculum learning format, training the second semantic classifier based on the initial data set and the first expanded subset in a first round of curriculum learning; training the second semantic classifier based on the initial data set, the first expanded subset, and the second expanded subset in a second round of curriculum learning; Including, The language processing method of claim 2 , wherein the third expanded subset and the fourth expanded subset are not for training the second semantic classifier.

4. generating a plurality of second predicted semantic labels associated with the expanded phrase from the second semantic classifier and a plurality of second confidence measures for the second predicted semantic labels; classifying the expanded phrases into a plurality of updated expanded subsets based on the second predicted semantic labels and the second confidence metric; training a third semantic classifier based on the updated expanded subset in a curriculum learning fashion; The language processing method according to claim 1 , further comprising:

5. The step of classifying the expanded phrases into the updated expanded subsets comprises: classifying a first portion of the expanded subset into a first updated expanded subset, where a comparison of the second predicted semantic label and the initial semantic label of the first portion shows a match and the second confidence measure of the first portion is greater than a first confidence threshold; classifying a second portion of the expanded subset into a second updated expanded subset, the second portion having the second predicted semantic label being consistent with the initial semantic label, and the second portion having the second confidence measure being lower than the first confidence threshold; classifying a third portion of the expanded subset into a third updated expanded subset, where a comparison result between the second predicted semantic label and the initial semantic label of the third portion indicates a mismatch and the second confidence measure of the third portion is higher than a second confidence threshold; classifying a fourth portion of the expanded subset into a fourth updated expanded subset, where a comparison result between the second predicted semantic label and the initial semantic label of the fourth portion indicates a mismatch and the second confidence measure of the fourth portion is lower than the second confidence threshold; The language processing method according to claim 4, comprising:

6. training the third semantic classifier based on the updated expanded subset in a curriculum learning fashion, training the third semantic classifier based on the initial data set and the first updated expanded subset in a first round of curriculum learning; training the third semantic classifier based on the initial data set, the first updated expanded subset, and the second updated expanded subset in a second round of curriculum learning; training the third semantic classifier based on the initial data set, the first updated expanded subset, the second updated expanded subset, and the third updated expanded subset in a third round of curriculum learning; Including, The language processing method of claim 5 , wherein the fourth updated expanded subset is not for training the third semantic classifier.

7. 7. The language processing method of claim 6, wherein the second predicted semantic labels associated with the expanded phrases generated by the second semantic classifier are used as a plurality of ground truths for training the third semantic classifier.

8. The step of performing sentence pattern expansion to generate the expanded phrase includes: A form (1) in which the initial phrase is rewritten using a large-scale language model to generate the expanded phrase; (2) using a translation model to translate the initial phrase expressed in a first language into a plurality of intermediate phrases expressed in a second language, and using the translation model to translate the intermediate phrase expressed in the second language into the expanded phrase expressed in the first language; (3) generating the expanded phrase by replacing a plurality of words in the initial phrase with a plurality of synonyms related to the words; or 2. The language processing method according to claim 1, further comprising: (4) inserting a plurality of random noises into the initial phrase to generate the expanded phrase.

9. The language processing method of claim 1 , further comprising generating an answer based on the meaning of the input phrase.

10. a storage unit for storing a plurality of computer executable instructions; a processing unit, coupled to the storage unit, that executes the computer-executable instructions to accomplish the following: obtaining an initial dataset including a plurality of initial phrases and a plurality of initial semantic labels associated with the initial phrases; training a first semantic classifier based on the initial dataset; performing sentence pattern expansion corresponding to the initial phrases to generate a plurality of expanded phrases; operating the first semantic classifier to generate a plurality of first predicted semantic labels associated with the expanded phrases and a plurality of first confidence measures for the first predicted semantic labels; classifying the expanded phrases into a plurality of expanded subsets based on a comparison result between the initial semantic labels and the first predicted semantic labels and the first confidence measure; and training a second semantic classifier for identifying the meaning of input phrases in a conversation based on a portion of the expanded subsets in a curriculum learning manner; A language processing system including

Images