A method, device, computer equipment and readable storage medium for voice input assessment of professional skills

Abstract

The present invention discloses a voice input assessment method, device, computer equipment and readable storage medium for occupational skills assessment, comprising: first, obtaining the test point configuration data of the target assessment scenario, including the standard answer text. Then, constructing a relevant sample data set, and training the enhanced SeACoParaformer architecture based on the data set to obtain a dedicated speech recognition model. Subsequently, the voice stream to be evaluated is input into the model for recognition to generate the corresponding text. Finally, the text to be evaluated is scored according to the standard answer text, and the keyword score and semantic similarity score are comprehensively considered as the final occupational skills assessment result. Such a design effectively improves the efficiency and accuracy of occupational skills assessment.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a voice input assessment method, device, computer equipment and readable storage medium for professional skill assessment. Background Art

[0002] With the development of technology, professional skills assessment has become an important way to measure an individual's professional ability. Traditional assessment methods mostly rely on written tests or practical operations, but these methods have limitations in terms of efficiency and objectivity. Summary of the Invention

[0003] The purpose of the present invention is to provide a voice input assessment method, device, computer equipment and readable storage medium for professional skills assessment.

[0004] In a first aspect, an embodiment of the present invention provides a voice input assessment method for professional skills assessment, comprising:

[0005] Obtaining test site configuration data for a target professional skills assessment scenario, wherein the test site configuration data includes standard answer text;

[0006] Constructing a sample dataset for the target occupational skills assessment scenario, and training the enhanced SeACoParaformer architecture based on the sample dataset to obtain a trained occupational skills assessment speech recognition model;

[0007] Inputting the speech stream to be evaluated into the professional skills assessment speech recognition model for recognition to obtain the text to be evaluated corresponding to the speech stream to be evaluated;

[0008] Scoring the text to be evaluated based on the standard answer text to obtain a keyword score and a semantic similarity score;

[0009] The keyword score and the semantic similarity score are used as the professional skill evaluation score corresponding to the speech stream to be evaluated.

[0010] In one possible implementation, the enhanced SeACoParaformer architecture includes a hot word component and a synonym enhancement component. Inputting the speech stream to be evaluated into the professional skills assessment speech recognition model for recognition to obtain the text to be evaluated corresponding to the speech stream to be evaluated includes:

[0011] The speech stream to be evaluated is optimized by the hot word component to obtain a text to be processed and input into the synonym enhancement component;

[0012] Performing synonym enhancement on the text to be processed based on the synonym enhancement component to obtain an enhanced text sequence;

[0013] The text sequence is used as the text to be evaluated corresponding to the speech stream to be evaluated.

[0014] In a possible implementation, performing synonym enhancement on the to-be-processed text based on the synonym enhancement component to obtain an enhanced text sequence includes:

[0015] Performing word segmentation processing on the text to be processed to obtain multiple word segments, and using a synonym dictionary to determine a synonym set for each word segment; using a pre-trained embedding table to retrieve the word segmentation embedding vector and the synonym set embedding vector corresponding to each word segment and each synonym set, and determining the position codes corresponding to the word segmentation embedding vector and the synonym set embedding vector, thereby realizing synonym embedding retrieval;

[0016] An attention mechanism is used to combine the word segmentation embedding vector, the synonym set embedding vector, and the position encoding corresponding to each of the word segmentation embedding vector and the synonym set embedding vector to determine an attention weight to achieve context fusion;

[0017] Performing weighted averaging on the synonym set embedding vectors based on the attention weights to obtain a fused synonym set embedding vector, thereby achieving word embedding fusion;

[0018] Calculate an enhanced word segmentation embedding vector for each word segmentation based on the word segmentation embedding vector, the word weight pre-configured for the word segmentation embedding vector, and the synonym set embedding vector to generate an enhanced vector;

[0019] Decoding the enhanced word segmentation embedding vector of each word segmentation in combination with the cross-attention mechanism to obtain a decoding state of the enhanced word segmentation embedding vector of each word segmentation, wherein the decoding state combines the self-attention mechanism and the cross-self-attention mechanism;

[0020] Mapping each of the decoding states to a vocabulary to obtain a score vector corresponding to the vocabulary for each of the segmented words; transforming the score vector using a softmax function to obtain a probability distribution of each segmented word corresponding to the vocabulary, and outputting an output segmented word for each segmented word included in the text to be processed based on the probability distribution to achieve vocabulary selection;

[0021] The output word segmentations are integrated to obtain the enhanced text sequence.

[0022] In a possible implementation, the test point configuration data includes multiple test question sections, and inputting the voice stream to be evaluated into the professional skills assessment speech recognition model for recognition to obtain the text to be evaluated corresponding to the voice stream to be evaluated includes:

[0023] In the target test question phase, the speech stream to be evaluated is input into the professional skill assessment speech recognition model for recognition, and at least one text to be evaluated corresponding to the speech stream to be evaluated is obtained;

[0024] The at least one text to be evaluated is integrated to obtain a text to be evaluated corresponding to the target test question segment.

[0025] In a possible implementation, scoring the text to be evaluated based on the standard answer text to obtain a keyword score and a semantic similarity score includes:

[0026] Based on the standard answer text, a sliding window of the text to be evaluated corresponding to the target test question is formed;

[0027] Based on the size of the sliding window, semantic segmentation is performed on the text to be evaluated corresponding to the target test question to obtain a plurality of short sentences to be evaluated;

[0028] Based on the standard answer text and each of the short sentences to be evaluated, scores are performed in turn to obtain keyword scores and semantic similarity scores.

[0029] In a possible implementation, the test point configuration further includes keywords, keyword weights, and a scoring strategy. The scoring based on the standard answer text and each of the short sentences to be evaluated is performed in sequence to obtain a keyword score and a semantic similarity score, including:

[0030] Obtaining a keyword scoring threshold and a semantic matching scoring threshold according to the scoring strategy;

[0031] Perform keyword matching on each of the short sentences to be evaluated according to the keywords and the keyword weights to obtain a pending keyword score;

[0032] In the case where the pending keyword score exceeds the keyword scoring threshold, the pending keyword score is used as the final keyword score;

[0033] Perform semantic matching on each of the short sentences to be evaluated according to the standard answer text to obtain a pending semantic similarity score;

[0034] In a case where the pending semantic similarity score exceeds the semantic matching score threshold, the pending semantic similarity score is used as the final semantic similarity score.

[0035] In one possible implementation, constructing a sample data set for the target occupational skill assessment scenario includes:

[0036] Filtering voice data and text data related to the target professional skill assessment scenario from a plurality of different professional skill assessment history records;

[0037] performing at least one data enhancement operation of adding noise, adjusting speech speed, and adjusting voice intonation on the filtered speech data;

[0038] The speech data after the data enhancement operation is combined with the corresponding text data to form the sample data set.

[0039] In a second aspect, an embodiment of the present invention provides a speech input assessment device for professional skills assessment, comprising:

[0040] An acquisition module is configured to obtain test site configuration data for a target vocational skill assessment scenario, the test site configuration data including standard answer text; construct a sample data set for the target vocational skill assessment scenario, and train the enhanced SeACoParaformer architecture based on the sample data set to obtain a trained vocational skill assessment speech recognition model; input a speech stream to be evaluated into the vocational skill assessment speech recognition model for recognition, and obtain the text to be evaluated corresponding to the speech stream to be evaluated;

[0041] An evaluation module is used to score the text to be evaluated based on the standard answer text to obtain a keyword score and a semantic similarity score; and use the keyword score and the semantic similarity score as the professional skill evaluation score corresponding to the voice stream to be evaluated.

[0042] In a third aspect, an embodiment of the present invention provides a computer device, comprising a processor and a non-volatile memory storing computer instructions, wherein when the computer instructions are executed by the processor, the computer device executes the method described in the first aspect.

[0043] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, wherein the readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method described in the first aspect.

[0044] Compared with the prior art, the beneficial effects provided by the present invention include: adopting a voice input evaluation method, device, computer equipment and readable storage medium for professional skills assessment disclosed by the present invention, by obtaining the test point configuration data of the target assessment scenario, including the standard answer text. Then, a relevant sample data set is constructed, and the enhanced SeACoParaformer architecture is trained based on the data set to obtain a dedicated speech recognition model. Subsequently, the voice stream to be evaluated is input into the model for recognition to generate the corresponding text. Finally, the text to be evaluated is scored according to the standard answer text, and the keyword score and semantic similarity score are comprehensively considered as the final professional skills assessment result. Such a design effectively improves the efficiency and accuracy of professional skills assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly describes the drawings required for use in the embodiments. It should be understood that the following drawings illustrate only certain embodiments of the present invention and should not be construed as limiting the scope of the present invention. Those skilled in the art can, without inventive effort, derive other relevant drawings from these drawings.

[0046] Figure 1 A schematic diagram of the steps of a voice input assessment method for professional skills assessment provided by an embodiment of the present invention;

[0047] Figure 2 A schematic block diagram of the structure of a voice input evaluation device for professional skills assessment provided by an embodiment of the present invention;

[0048] Figure 3 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the described embodiments are only a portion of the embodiments of the present invention, not all of them. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0050] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0051] In order to solve the technical problems in the above background technology, Figure 1 This is a flow chart of a voice input assessment method for professional skills assessment provided in an embodiment of the present disclosure. The voice input assessment method for professional skills assessment is introduced in detail below.

[0052] Step S201: Acquire test site configuration data for a target professional skill assessment scenario, wherein the test site configuration data includes a standard answer text;

[0053] Step S202: constructing a sample dataset for the target occupational skill assessment scenario, and training the enhanced SeACoParaformer architecture based on the sample dataset to obtain a trained occupational skill assessment speech recognition model;

[0054] Step S203: inputting the speech stream to be evaluated into the professional skills assessment speech recognition model for recognition to obtain the text to be evaluated corresponding to the speech stream to be evaluated;

[0055] Step S204, scoring the text to be evaluated based on the standard answer text to obtain a keyword score and a semantic similarity score;

[0056] Step S205: Using the keyword score and the semantic similarity score as the professional skill evaluation score corresponding to the speech stream to be evaluated.

[0057] In the embodiment of the present invention, for example, in the context of a professional skills assessment, the server is responsible for managing the entire assessment process and processing data. Taking the chef professional skills assessment as an example, when conducting a specific chef skills assessment, the target professional skills assessment scenario is the chef skills assessment scenario.

[0058] The server first needs to obtain the test site configuration data for this scenario. This data contains a variety of important information, the most important of which is the standard answer text. For the chef skill assessment scenario, there will be different assessment stages, such as ingredient preparation, cooking techniques, and dish seasoning.

[0059] In the ingredient preparation phase, the standard answer text will describe in detail the correct ingredient selection criteria. For example, "For a classic French steak, grain-fed beef tenderloin should be used, weighing between 200-250 grams. The freshness of the meat should ensure that the surface is shiny, odorless, and elastic when pressed." In the cooking technique phase, the standard answer text will mention things like "When frying steak, first heat the pan over high heat, put the steak in, and fry each side for 2-3 minutes to ensure a golden crust forms on the surface, then turn to low heat and continue frying until the inside reaches the desired degree of doneness (such as the corresponding internal temperature requirements for medium-rare, medium-rare, etc.)." These standard answer texts are an important basis for subsequent evaluation. The server obtains these test point configuration data through a pre-configured database or from a dedicated evaluation standard management module.

[0060] The server filters data from numerous historical records of professional skill assessments. For example, for a chef's skill assessment, the server searches through previous chef skill assessment records to find voice and text data relevant to the current target scenario (e.g., a French chef's skill assessment). This history might include voice responses from different chefs during previous assessments, along with corresponding standard text responses.

[0061] For example, a chef might describe the process of frying a steak in a voice recording, saying, "I first heat the pan, then put the steak in, fry it for a while, and then flip it over." The corresponding standard text answer is the aforementioned precise steak-frying process. The server will filter out these relevant voice and text data.

[0062] The server will perform data enhancement operations on the filtered voice data.

[0063] Noise addition: This function simulates the noise interference of a real kitchen environment, such as adding the sounds of a burning stove or a range hood. For example, the originally clear voice of a chef describing the steps of frying a steak, "I first heat the pan, then put the steak in, fry it for a while, and then flip it over," will become a voice with some noisy background sound after adding noise, but the content remains unchanged.

[0064] Speech speed adjustment: Chefs may speak at different speeds during cooking due to nervousness or varying levels of proficiency. The server will speed up or slow down the normal speech, for example, adjusting the above speech to a faster "I'll heat the pan first, then put the steak in and sear it for a while, then flip it over" or a slower "I'll slowly heat the pan, then carefully put the steak in, sear it for a while, and then flip it over."

[0065] Tone adjustment: Different chefs have different tones when expressing themselves. Servers will adjust the tone of their voices, such as changing a plain tone to an emphatic tone to emphasize certain key steps.

[0066] The augmented speech data is combined with the corresponding text data. For example, the processed speech data describing the steps for frying a steak is combined with the corresponding standard text answers to form part of the sample dataset. This sample dataset covers a combination of speech and text data from multiple steps and situations in a chef's skill assessment.

[0067] The server uses the constructed sample dataset to train the enhanced SeACoParaformer architecture. This architecture has some special components, such as the hot word component and the synonym enhancement component. During the training process, the speech data in the sample dataset is used as input, and the corresponding text data is used as the output target.

[0068] For example, for a large amount of speech data containing vocabulary related to chef skills (hot words such as frying, stir-frying, grilling, and ingredient names), the hot word component can learn the characteristic patterns of these hot words in speech, so as to more accurately identify these key hot words in subsequent recognition. The synonym enhancement component helps to handle the different expressions that chefs may use in the description process. For example, "fried steak" may also be said as "steak fried in oil." By learning the synonym relationships in the sample dataset, the model can better understand the meaning of different expressions and accurately identify them. After multiple iterative training until the model converges, the trained professional skills assessment speech recognition model is obtained.

[0069] Imagine a chef undergoing a skills assessment. He describes his French steak cooking process into a microphone. The server receives this speech stream for evaluation. The server then feeds it into a previously trained speech recognition model for professional skills assessment. The model recognizes the speech based on learned speech features and language patterns and converts the speech into text. For example, if the chef says, "I'll heat the pan first, then add the steak I've chosen. When it turns golden brown, I'll flip it." The model recognizes the text and generates the corresponding text: "I'll heat the pan first, then add the steak I've chosen. When it turns golden brown, I'll flip it."

[0070] If the speech stream to be evaluated contains hot words in chef skill evaluation scenarios, such as "steak" and "frying," the hot word component will first optimize these hot words. For example, if a chef says "steak" with a heavy accent, the hot word component can accurately identify it as "steak" based on pre-learned hot word features, and then generate the hot word-optimized text to be processed.

[0071] Next, the text to be processed is input into the synonym enhancement component.

[0072] Determining word segments and synonym sets: For the text to be processed, "I first heat the pan, then put the selected steak in, and flip it when it turns golden brown," we perform word segmentation to obtain multiple segmented words, such as "I first heat the pan," "then," "put the selected steak in," "fry," "when," "see," "the color," "turn," "golden brown," and "flip." A synonym dictionary is used to determine the synonym set for each segmented word. For example, synonyms for "heat" might include "heat to very hot" and "burn," while synonyms for "flip" might include "turn over" and "turn the noodles over."

[0073] Synonym embedding retrieval: Use a pre-trained embedding table to retrieve the word segment embedding vector and synonym set embedding vector corresponding to each word segment and each synonym set, and determine their corresponding positional encodings. For example, for the word "burn hot" and its synonym set "heat to very hot", their embedding vectors and positional encodings are obtained respectively.

[0074] Contextual fusion: The attention mechanism combines word segmentation embedding vectors, synonym set embedding vectors, and their corresponding positional encodings to determine attention weights. For example, in the context of this chef skill description, the word "heat" is closely related to the subsequent step of "frying the steak." The attention mechanism assigns appropriate weights to this word and its synonyms in this context, achieving contextual fusion.

[0075] Word embedding fusion: The weighted average of the synonym set embedding vectors is performed based on the attention weights to obtain the fused synonym set embedding vector. For example, the embedding vectors of the synonym set of "burning" are weighted averaged based on the calculated attention weights to obtain the fused vector.

[0076] Enhanced vector generation: Based on the word embedding vector, the pre-configured word weights, and the synonym set embedding vector, an enhanced word embedding vector is calculated for each word. For example, for the word "burning", the enhanced embedding vector is calculated based on the previously obtained information.

[0077] Decoding: The enhanced word embedding vector of each word is decoded using the cross-attention mechanism to obtain the decoded state of the enhanced word embedding vector of each word. This decoded state combines the self-attention mechanism and the cross-self-attention mechanism. For example, for the word "burning hot", the decoded state contains more semantic information.

[0078] Vocabulary selection: Map each decoding state to a vocabulary, obtaining a score vector for each word in the vocabulary. Use the softmax function to transform the score vector to obtain a probability distribution for each word in the vocabulary. Based on this probability distribution, the output word for each word in the processed text is output. For example, for the word "burn hot," the probability distribution is used to select the output word that best matches the semantics (it could still be "burn hot" or the most appropriate synonym).

[0079] Integration to obtain the text to be evaluated: Finally, all the output segmentations are integrated to obtain the enhanced text sequence, which serves as the text to be evaluated corresponding to the speech stream to be evaluated.

[0080] If the chef's speech flow is long or intermittent during a particular test segment, at least one text to be evaluated will be generated. For example, the chef's description of the steak-making process is split into two parts. The server recognizes the two texts to be evaluated: "I first heat the pan, then add the steak" and "Flip it when it turns golden brown." The server integrates these at least one text to be evaluated to generate the text to be evaluated for this test segment: "I first heat the pan, then add the steak, and flip it when it turns golden brown."

[0081] For example, in the steak-making step of a chef's skill assessment, based on the standard answer, the server creates a sliding window for the text to be assessed for this step. Let's assume the standard answer is, "I first heat the pan, then add a grain-fed tenderloin steak weighing 200-250 grams and of acceptable freshness. I fry it over high heat until a golden crust forms (2-3 minutes per side), then reduce the heat to low and fry until the desired doneness is achieved, flipping it once it turns golden."

[0082] If the sliding window size is set to 5 word segmentation lengths, the server will perform semantic segmentation on the text to be evaluated based on this size: "I first heat the pan, then put the selected steak in, and flip it when it turns golden brown." This will result in multiple short sentences to be evaluated, such as "I first heat the pan," "Then put the selected steak in," and "Flip it when it turns golden brown."

[0083] Keyword matching: The test point configuration also includes keywords, keyword weights, and scoring strategies. Assuming that the keywords include "steak", "pan", "fried", "golden yellow", etc., the keyword weights are set to 0.2, 0.1, 0.3, 0.2, etc. according to their importance. For each short sentence to be evaluated, keyword matching is performed based on the keywords and keyword weights. For example, for the short sentence "I heat the pan first", which contains the keyword "pan", a pending keyword score (such as 0.1) is calculated based on the weight. If the pending keyword score exceeds the keyword scoring threshold (assuming it is 0.05), the pending keyword score is used as the final keyword score.

[0084] Semantic matching: Perform semantic matching on each short sentence to be evaluated based on the standard answer text to obtain a pending semantic similarity score. For example, for the short sentence "I heat the pot first," perform semantic similarity calculations with the corresponding part of the standard answer text (using some semantic similarity algorithms, such as cosine similarity), and obtain a pending semantic similarity score (e.g., 0.8). If this pending semantic similarity score exceeds the semantic matching score threshold (assuming it is 0.6), this pending semantic similarity score is used as the final semantic similarity score.

[0085] After the previous steps, the server obtains a keyword score and a semantic similarity score. For this chef's skill evaluation, let's assume the keyword score is 0.6 and the semantic similarity score is 0.7. The server uses these two scores as the chef's professional skill assessment score for the speech stream to be evaluated in this steak-making step. This score serves as an important basis for a comprehensive assessment of the chef's skills. It is aggregated with the scores of other steps to ultimately determine the chef's overall skill score.

[0086] In an embodiment of the present invention, the enhanced SeACoParaformer architecture includes a hot word component and a synonym enhancement component. The voice stream to be evaluated is input into the professional skill assessment speech recognition model for recognition to obtain the text to be evaluated corresponding to the voice stream to be evaluated, which can be implemented through the following example.

[0087] The speech stream to be evaluated is optimized by the hot word component to obtain a text to be processed and input into the synonym enhancement component;

[0088] Performing synonym enhancement on the text to be processed based on the synonym enhancement component to obtain an enhanced text sequence;

[0089] The text sequence is used as the text to be evaluated corresponding to the speech stream to be evaluated.

[0090] In this embodiment of the present invention, for example, it is assumed that the current professional skills assessment scenario is a doctor's clinical skills assessment. The speech stream to be assessed is a speech recording of a doctor describing the process of diagnosing pneumonia on a patient.

[0091] In a medical setting, hot words might include "pneumonia," "auscultation," "X-ray," "cough," "fever," and other words closely related to pneumonia diagnosis. When the server receives the doctor's voice stream to be evaluated, the hot word component begins to work.

[0092] For example, a doctor might pronounce "pneumonia" as "feiyan" due to their accent or rapid speech. Based on the pre-trained hotword patterns, the hotword component can identify this mispronunciation and correct it to "pneumonia." Another example is a doctor saying, "I first performed auscultation." Based on the phonetic characteristics of the hotword "auscultation," the hotword component can correct "listening rash" to "auscultation." After optimizing the hotword component, the resulting text is processed, such as "I diagnosed pneumonia in this patient with cough and fever, first performing auscultation and then examining the X-ray."

[0093] The server then feeds the text to be processed, optimized by the hotword component, into the synonym enhancement component. This process is like passing the initially organized information to the next, more refined processing unit, paving the way for further exploration of the text's semantic information.

[0094] For the input text to be processed, "I diagnosed pneumonia on this patient with cough and fever symptoms by first performing auscultation and then checking the X-ray film," the synonym enhancement component first performs word segmentation processing.

[0095] The resulting segmented words include "I," "to," "this person," "who has," "cough," "and," "fever," "symptoms," "of," "patient," "diagnosed," "first," "conducted," "auscultation," and "then," "checked," and "X-rays." A synonym dictionary is then used to determine the synonym set for each segmented word.

[0096] For example, the synonym set of "cough" may include "dry cough", "coughing up phlegm", etc. (in some cases, doctors will describe the type of cough more specifically); the synonym set of "fever" may include "fever" and "elevated body temperature"; the synonym set of "diagnosis" may include "judgment" and "diagnosis"; the synonym set of "check" may include "observation" and "review", etc.

[0097] The pre-trained embedding table is used to retrieve the word segmentation embedding vector and synonym set embedding vector corresponding to each word segment and each synonym set, and their respective corresponding position encodings are determined.

[0098] Taking the word "cough" as an example, the word embedding vector corresponding to "cough" is found in the pre-trained embedding table. This vector can represent the semantic characteristics of the word "cough" in vector space. Simultaneously, for words in the synonym set, such as "dry cough" and "cough up phlegm," the corresponding synonym set embedding vectors are also found. Furthermore, based on the position of these words in the text being processed, their corresponding positional encodings are determined. Positional encodings reflect the order of words in a sentence, which is crucial for understanding semantics.

[0099] The attention weight is determined by combining the word segmentation embedding vector, the synonym set embedding vector, and their corresponding position encoding.

[0100] In the text to be processed in this medical scenario, "cough" and "fever" are patient symptoms and are closely related to "pneumonia diagnosis." The attention mechanism takes this semantic relationship into account when calculating attention weights. For example, the association weight between the word "cough" and "pneumonia diagnosis" is relatively high because cough is a common symptom of pneumonia. Similarly, "auscultation" is strongly associated with "pneumonia diagnosis." Therefore, when calculating attention weights, these related word segments and synonyms are given appropriate weights, achieving contextual integration.

[0101] The synonym set embedding vectors are weighted averaged based on the attention weights to obtain the fused synonym set embedding vectors.

[0102] Continuing with the example of "cough," if the synonym sets for "cough," "dry cough" and "coughing up phlegm," have attention weights of 0.3 and 0.2, respectively (assuming these weights), then the synonym embedding vectors for "dry cough" and "coughing up phlegm" are weighted averaged based on these weights to obtain the fused synonym embedding vector. This fused vector comprehensively considers the importance of synonyms in a specific context and more accurately represents the semantic information of "cough" and its synonyms in this medical scenario.

[0103] The enhanced word segmentation embedding vector of each word is calculated based on the word segmentation embedding vector, the pre-configured word weight of the word segmentation, and the synonym set embedding vector.

[0104] For the word "pneumonia," it has a pre-configured word weight (assuming it is 0.5, as pneumonia is a core word in this diagnostic scenario). This word embedding vector is combined with information about synonyms (if any) to calculate an enhanced word embedding vector. This enhanced vector more comprehensively represents the semantic information of the word "pneumonia" in the current medical scenario, taking into account the importance of the word itself and possible synonyms.

[0105] Combined with the cross-attention mechanism, the enhanced word segmentation embedding vector of each word is decoded to obtain the decoding state of the enhanced word segmentation embedding vector of each word. This decoding state combines the self-attention mechanism and the cross-self-attention mechanism.

[0106] For example, for the word "auscultation," its enhanced word embedding vector interacts with other related words (such as "pneumonia" and "patient") through a cross-attention mechanism. It also uses a self-attention mechanism to focus on its own internal semantic information, and a cross-self-attention mechanism to further integrate its relationships with other words, resulting in a decoding state rich in semantic information. This decoding state better reflects the semantic role of "auscultation" in the entire pneumonia diagnosis process.

[0107] Map each decoding state to the vocabulary to obtain the score vector of each word segmentation corresponding to the vocabulary; use the softmax function to transform the score vector to obtain the probability distribution of each word segmentation corresponding to the vocabulary, and output the output word segmentation of each word segmentation included in the text to be processed based on the probability distribution.

[0108] For example, the word "view" (see) is decoded and mapped to the vocabulary, resulting in a score vector. This score vector represents the similarity between "view" and each word in the vocabulary. The softmax function is then used to convert this score vector into a probability distribution. Assume that in this probability distribution, "view" itself has the highest probability, but "observe" also has a certain probability. Based on this probability distribution, the final word output might still be "view" (if its probability is high enough), but it might also be "observe" if the probability of "observe" meets certain requirements.

[0109] Finally, all the output segmented words are integrated to obtain an enhanced text sequence. For example, after the above processing, the output segmented words are "I", "to", "this person", "have", "dry cough" (if "dry cough" is selected in the vocabulary selection), "and", "fever" (if "fever" is selected), "symptoms", "of", "patient", "underwent", "pneumonia", "diagnosed" (if "diagnosed" is selected), "first", "underwent", "auscultation", "then", "observed" (if "observed" is selected), and "X-ray". This enhanced text sequence, which serves as the text to be evaluated corresponding to the speech stream to be evaluated, is more accurate and rich in semantic expression than the original processed text, and can provide higher-quality input for the subsequent scoring process.

[0110] In the embodiment of the present invention, the synonym enhancement component is used to perform synonym enhancement on the text to be processed to obtain an enhanced text sequence, which can be implemented through the following examples.

[0111] Performing word segmentation processing on the text to be processed to obtain multiple word segments, and using a synonym dictionary to determine a synonym set for each word segment; using a pre-trained embedding table to retrieve the word segmentation embedding vector and the synonym set embedding vector corresponding to each word segment and each synonym set, and determining the position codes corresponding to the word segmentation embedding vector and the synonym set embedding vector, thereby realizing synonym embedding retrieval;

[0112] An attention mechanism is used to combine the word segmentation embedding vector, the synonym set embedding vector, and the position encoding corresponding to each of the word segmentation embedding vector and the synonym set embedding vector to determine an attention weight to achieve context fusion;

[0113] Performing weighted averaging on the synonym set embedding vectors based on the attention weights to obtain a fused synonym set embedding vector, thereby achieving word embedding fusion;

[0114] Calculate an enhanced word segmentation embedding vector for each word segmentation based on the word segmentation embedding vector, the word weight pre-configured for the word segmentation embedding vector, and the synonym set embedding vector to generate an enhanced vector;

[0115] Decode the enhanced token embedding vectors of each of the tokens by combining a cross-attention mechanism to obtain the decoding states of the enhanced token embedding vectors of each of the tokens, where the decoding states combine a self-attention mechanism and a cross self-attention mechanism;

[0116] Map each of the decoding states to a vocabulary to obtain a score vector of each of the tokens corresponding to the vocabulary; use the softmax function to transform the score vector to obtain a probability distribution of each of the tokens corresponding to the vocabulary, and output an output token of each of the tokens included in the text to be processed based on the probability distribution, thereby implementing vocabulary selection;

[0117] Integrate the output tokens to obtain the enhanced text sequence.

[0118] In an embodiment of the present invention, by way of example, it is assumed that in a scenario of evaluating teachers' teaching skills, the text to be processed is "When a teacher is telling a historical event, multimedia materials are used to assist teaching, and this method can enable students to better understand complex historical plots."

[0119] The server performs tokenization on this text to be processed, and the tokens obtained are "teacher", "when", "telling", "historical event", "when", "used", "multimedia materials", "to", "assist teaching", "this method", "can", "enable", "students", "better", "understand", "complex", "historical plots".

[0120] Then, use a thesaurus to determine the synonym set of each token. For example, the synonym set of "telling" may include "lecturing", "expounding"; the synonym set of "multimedia materials" may be "multimedia materials", "multimedia resources"; the synonym set of "assist teaching" may include "assist in teaching", "facilitate teaching"; the synonym set of "understand" may be "comprehend", "grasp", etc.

[0121] For the tokens and synonym sets in the above educational scenario, the server uses a pre-trained embedding table for retrieval.

[0122] Taking the token "telling" as an example, in the pre-trained embedding table, there is a specific vector to represent the semantic feature of "telling", which is the token embedding vector. For the words in the synonym sets such as "lecturing" and "expounding", there are also corresponding synonym set embedding vectors.

[0123] At the same time, position codes are determined. In the text to be processed, "When the teacher recounted historical events, he used multimedia materials to assist teaching, which helped students better understand complex historical plots," "narrate" is in the third position. Its position code is determined based on predefined coding rules. Similarly, position codes for synonyms such as "lecture" and "explain" are determined based on their possible positional relationships within the text.

[0124] In this educational scenario, “narration” is semantically related to “historical events” and other participles such as “students” and “understanding”.

[0125] The attention mechanism combines the word embedding vector of "narrate," the synonym set embedding vectors of its synonyms ("lecture," "explain"), and their respective positional encodings to determine attention weights. For example, because "narrate" is closely related to "historical events," this relationship is taken into account when calculating attention weights, giving "narrate" and its synonyms appropriate weights in this context. If the word "narrate" is more important in its association with "historical events," then its attention weight in this semantic relationship will be relatively high, achieving contextual integration.

[0126] Taking the synonym set of "tell" "teach" and "explain" as an example, assume that the attention weight of "teach" is determined to be 0.4 and the attention weight of "explain" is determined to be 0.6 through the attention mechanism.

[0127] The server then takes a weighted average of the synonym embedding vectors for "lecture" and "elaboration" based on these weights. If the synonym embedding vector for "lecture" is vector A, and the synonym embedding vector for "elaboration" is vector B, then the fused synonym embedding vector = 0.4 * vector A + 0.6 * vector B. This yields a fused synonym embedding vector that comprehensively considers contextual relationships.

[0128] For the word "narrate", assume that its pre-configured word weight is 0.3 (because narration is a relatively important behavior in the teaching scenario).

[0129] The word segmentation embedding vector is vector C, and the fused synonym set embedding vector is the vector calculated previously (assuming it is vector D). The enhanced word segmentation embedding vector for "narrate" is calculated according to the formula: 0.3 * vector C + (1 - 0.3) * vector D. This enhanced word segmentation embedding vector combines the importance of the word itself and the information about synonyms, providing a more comprehensive representation of the semantics of "narrate" in this educational context.

[0130] Taking the word "assisted teaching" as an example, its enhanced word segmentation embedding vector contains rich semantic information related to synonyms.

[0131] The server decodes the word using a cross-attention mechanism. This mechanism interacts with the enhanced word embedding vector for "assisted teaching" and the enhanced word embedding vectors for other related words (such as "teacher," "student," and "multimedia materials"). Simultaneously, the self-attention mechanism focuses on the semantic structure of "assisted teaching," and the cross-self-attention mechanism further integrates its complex relationships with other words. Through the synergistic effect of these mechanisms, the decoded state of the enhanced word embedding vector for "assisted teaching" is obtained, which incorporates a deeper understanding of the semantics.

[0132] Taking the word "understanding" as an example, after its decoding state is mapped to a vocabulary (this vocabulary contains many words in educational contexts), a score vector is obtained. This score vector represents the similarity score between "understanding" and each word in the vocabulary.

[0133] The softmax function is then used to convert this score vector into a probability distribution. Assume that in this probability distribution, "understanding" itself has the highest probability, but "comprehension" also has a certain probability. If this probability distribution meets certain conditions (for example, the probability of "comprehension" exceeds a certain threshold or the probability of "comprehension" is below a certain lower limit), then "comprehension" may be output as the output word for this segmentation; otherwise, "comprehension" is output.

[0134] After the vocabulary selection, each word segmentation is output. For example, the word "narrate" in the original text to be processed may be output as "lecture" (if selected in the vocabulary selection), and "multimedia material" may be output as "multimedia resource", etc.

[0135] The server integrates all the output segmented words to generate an enhanced text sequence, such as "When teaching historical events, the teacher used multimedia resources to assist in teaching, which helped students better grasp the complex historical plot." This enhanced text sequence is more accurate and rich in semantic expression than the original processed text, better reflecting the teacher's behavior and purpose during the teaching process, providing a better basis for subsequent evaluation.

[0136] In an embodiment of the present invention, the test point configuration data includes multiple test question links, and the voice stream to be evaluated is input into the professional skill assessment voice recognition model for recognition to obtain the text to be evaluated corresponding to the voice stream to be evaluated. This can be implemented through the following example.

[0137] In the target test question phase, the speech stream to be evaluated is input into the professional skill assessment speech recognition model for recognition, and at least one text to be evaluated corresponding to the speech stream to be evaluated is obtained;

[0138] The at least one text to be evaluated is integrated to obtain a text to be evaluated corresponding to the target test question segment.

[0139] In this embodiment of the present invention, for example, assume that an automobile maintenance skills assessment is being conducted. This assessment includes multiple test sections, such as automobile fault diagnosis, parts replacement, and repair tool use. Now, let's focus on the target test section, automobile fault diagnosis.

[0140] When the repair technician describes his car fault diagnosis process into the microphone, the server receives the speech stream to be evaluated and inputs it into the pre-trained professional skills assessment speech recognition model.

[0141] Because technicians may interrupt their descriptions or explain their fault diagnosis ideas from different perspectives, the model will receive at least one text to be evaluated. For example, if a technician first says, "This car has difficulty starting, and the engine vibrates abnormally when running. I initially suspect a spark plug problem, as they are quite old," the model will recognize the text and receive the following text to be evaluated: "This car has difficulty starting, and the engine vibrates abnormally when running. I initially suspect a spark plug problem, as they are quite old."

[0142] The technician then added, "I checked the ignition coil and found that the connection line of the ignition coil is not loose, so it is basically confirmed that the spark plug is faulty." The model again recognized another text to be evaluated: "I checked the ignition coil and found that its connection line is not loose, so it is basically confirmed that the spark plug is faulty."

[0143] Taking software programming skills assessment as an example, one of the target test questions is algorithm design.

[0144] When a programmer describes his sorting algorithm, the server receives the corresponding speech stream to be evaluated and inputs it into the speech recognition model. The programmer might first describe the basic idea of ​​the algorithm: "I have designed an improved bubble sort algorithm. Unlike traditional bubble sort, it skips the comparison of adjacent elements if they are already in order, which improves efficiency." The model recognizes the text to be evaluated: "I have designed an improved bubble sort algorithm. Unlike traditional bubble sort, it skips the comparison of adjacent elements if they are already in order, which improves efficiency."

[0145] Afterwards, the programmer further explained the time complexity of the algorithm: "In the best case, the time complexity of this algorithm can reach O(n), because when the elements are basically ordered, most comparison operations can be skipped." The model then identified a text to be evaluated: "The best case time complexity of the algorithm is O(n), and most comparison operations can be skipped when the elements are basically ordered."

[0146] In the automobile repair fault diagnosis phase, the server needs to integrate at least one previously obtained text to be evaluated.

[0147] The server integrates the two previously mentioned texts to be evaluated regarding automobile fault diagnosis: "This car has difficulty starting and there is abnormal vibration when the engine is running. We initially suspect a problem with the spark plugs because they have been used for a long time" and "Check the ignition coil and its connection line is not loose. It is basically confirmed that the spark plug is faulty."

[0148] The server will integrate according to semantic logic. For example, it will arrange them in the order of the normal fault diagnosis process to obtain the integrated text to be evaluated: "This car is difficult to start, and there is abnormal shaking when the engine is running. It was initially suspected to be a problem with the spark plugs because they have been used for a long time. After checking the ignition coil, it was found that the connection line of the ignition coil was not loose, so it was basically determined that it was a spark plug failure."

[0149] In the algorithm design phase of software programming, the server integrates the two previously obtained texts to be evaluated.

[0150] Integrate "designing an improved bubble sort algorithm, which is different from the traditional one and skips comparison when adjacent elements are already in order to improve efficiency" and "the best-case time complexity of the algorithm is O(n), and most comparison operations can be skipped when the elements are basically in order."

[0151] According to the logical order described in the programming algorithm, the server integrates the text to be evaluated as "I designed an improved bubble sort algorithm. Unlike the traditional bubble sort, each time it compares adjacent elements, if the elements are already in order, the comparison is skipped, which can improve efficiency. In the best case, the time complexity of this algorithm can reach O(n) because most comparison operations can be skipped when the elements are basically in order."

[0152] Through this integration process, a complete, semantically coherent assessment text is generated for each target question, enabling subsequent scoring based on the standard answer text. This integration provides a more comprehensive and accurate reflection of the examinee's performance in a specific question segment, avoiding information fragmentation caused by speech interruptions or presentation from different perspectives.

[0153] In an embodiment of the present invention, the text to be evaluated is scored based on the standard answer text to obtain a keyword score and a semantic similarity score, which can be implemented through the following examples.

[0154] Based on the standard answer text, a sliding window of the text to be evaluated corresponding to the target test question is formed;

[0155] Based on the size of the sliding window, semantic segmentation is performed on the text to be evaluated corresponding to the target test question to obtain a plurality of short sentences to be evaluated;

[0156] Based on the standard answer text and each of the short sentences to be evaluated, scores are performed in turn to obtain keyword scores and semantic similarity scores.

[0157] In the embodiment of the present invention, for example, it is assumed that in the construction skills assessment of a building project, the target test question link is the concrete pouring link.

[0158] The standard answer text is "Before pouring concrete, check the formwork and steel bars to ensure that the formwork is firmly installed and without deformation, and that the specifications, quantity and position of the steel bars meet the design requirements. Then, use appropriate pouring equipment, such as a concrete pump truck or hopper, to pour the concrete evenly into the formwork. During the pouring process, pay attention to vibration to make the concrete dense and avoid defects such as honeycomb surfaces. The pouring height should not be too high, and it should be poured in layers, with the thickness of each layer controlled between 30-50 cm."

[0159] Based on this standard answer, the server forms a sliding window for the text to be evaluated in this step. Assume that the sliding window is set to 15 Chinese characters (this size can be set based on actual needs and language expression habits). This sliding window acts as a specific perspective for observing the text to be evaluated, which is used for subsequent semantic segmentation and scoring operations.

[0160] For the text to be evaluated in which construction workers describe the operation process during the concrete pouring phase, it is assumed to be "I first checked the formwork and steel bars. The formwork was installed firmly, and the specifications, quantity and position of the steel bars were correct. Then I used a concrete pump truck to pour concrete into the formwork. During pouring, I vibrated the concrete to make it dense, and poured it in layers. The thickness of each layer was about 40 cm."

[0161] The server performs semantic segmentation on the text to be evaluated based on the sliding window size (15 Chinese characters). The first sentence to be evaluated is "I first checked the formwork and rebar, and the formwork is firmly installed." This sentence fits perfectly within the sliding window and fully expresses an operation step. The next sentence corresponding to the sliding window is "The specifications, quantity, and location of the rebar are correct. Then, a concrete pump truck is used to pour concrete into the formwork." Although this sentence spans parts of two sentences, the sliding window segmentation yields a relatively complete sentence. Similarly, further segmentation yields the following sentences: "Vibrate to compact the concrete during pouring, and pour in layers" and "Each layer is approximately 40 cm thick."

[0162] In the configuration of test points for the concrete pouring link of construction projects, the keywords include "formwork", "rebar", "concrete pump truck", "vibration", "layered pouring", etc. The keyword weights are set according to their importance. For example, the weights of "formwork" and "rebar" are 0.15 (because the preliminary inspection of formwork and rebar is an important prerequisite for pouring), the weight of "concrete pump truck" is 0.1 (as pouring equipment, it is part of the pouring operation), the weight of "vibration" is 0.2 (vibration has a great influence on the density of concrete), and the weight of "layered pouring" is 0.2. The scoring strategy is set to get a score of the corresponding weight if the keyword is fully matched, a score of half the weight if it is partially matched, and 0 points if it is not matched.

[0163] The sentence to be evaluated, "I first checked the formwork and steel bars, and the formwork was installed firmly," contains the keywords "formwork" and "steel bars," which are a perfect match. Therefore, the scores of these two keywords are 0.15 respectively. The score of the pending keywords for this sentence is 0.15+0.15=0.3.

[0164] For the short sentence "The specifications, quantity and position of the steel bars are correct. Then use a concrete pump truck to pour the concrete into the formwork", the keywords "steel bars", "concrete pump truck" and "formwork" are all fully matched. The score of "steel bars" is 0.15, the score of "concrete pump truck" is 0.1, and the score of "formwork" is 0.15. The score of the pending keyword is 0.15+0.1+0.15=0.4.

[0165] The server uses a predefined semantic similarity algorithm, such as a cosine similarity algorithm, to calculate the semantic similarity between each short sentence to be evaluated and the corresponding part of the standard answer text.

[0166] For the short sentence to be evaluated, "I first checked the formwork and rebar. The formwork is securely installed." Compute semantic similarity between it and the corresponding portion of the standard answer, "Before pouring concrete, check the formwork and rebar to ensure they are securely installed and free of deformation, and that the specifications, quantity, and location of the rebar meet design requirements." Assume the semantic similarity calculated using the cosine similarity algorithm is 0.85. If the semantic matching score threshold is set to 0.7, since 0.85 > 0.7, the semantic similarity score for this short sentence is 0.85.

[0167] The semantic similarity calculation for the sentence "Each layer is approximately 40 cm thick" and the corresponding portion of the standard answer text, "The pouring height should not be too high; it should be poured in layers, with each layer thickness controlled between 30 and 50 cm," yields a semantic similarity score of 0.9. Because 0.9 > 0.7 (the semantic matching score threshold), the semantic similarity score for this sentence is 0.9.

[0168] Through the above steps, the server scores each short sentence to be evaluated based on the standard answer text in turn, and obtains the keyword score and semantic similarity score of each short sentence. Then, it can conduct a comprehensive evaluation of the performance of the entire text to be evaluated in the target test question link, providing accurate data support for the final professional skills assessment.

[0169] In an embodiment of the present invention, the test point configuration also includes keywords, keyword weights and scoring strategies. The standard answer text is scored in turn with each of the short sentences to be evaluated to obtain keyword scores and semantic similarity scores, which can be implemented through the following examples.

[0170] Obtaining a keyword scoring threshold and a semantic matching scoring threshold according to the scoring strategy;

[0171] Perform keyword matching on each of the short sentences to be evaluated according to the keywords and the keyword weights to obtain a pending keyword score;

[0172] In the case where the pending keyword score exceeds the keyword scoring threshold, the pending keyword score is used as the final keyword score;

[0173] Perform semantic matching on each of the short sentences to be evaluated according to the standard answer text to obtain a pending semantic similarity score;

[0174] In a case where the pending semantic similarity score exceeds the semantic matching score threshold, the pending semantic similarity score is used as the final semantic similarity score.

[0175] In an embodiment of the present invention, illustratively, in the electrical engineering installation skills assessment, the target test question link is the laying of electrical lines.

[0176] The server obtains information related to the scoring strategy from the test center configuration. For example, the scoring strategy specifies the basis for setting the keyword scoring threshold and the semantic matching scoring threshold. Assuming that in this scenario, the keyword scoring threshold is set to 0.3, which means that if the keyword matching score in a short sentence to be evaluated is lower than 0.3, the keyword matching degree is considered insufficient; the semantic matching scoring threshold is set to 0.6, that is, when the semantic similarity between the short sentence to be evaluated and the standard answer text is lower than 0.6, it indicates that the semantic matching degree is poor. These thresholds are determined based on multiple factors such as the requirements of the electrical engineering installation skills assessment, previous assessment experience, and expectations for the level of skill mastery.

[0177] For the electrical line laying process, the keywords in the test point configuration include "electrical wires", "wire pipes", "junction boxes", "bridges", "grounding", etc. The keyword weights are set according to their importance in the laying of electrical lines. For example, the weight of "electrical wires" is 0.25 (because electric wires are the core elements of line laying), the weight of "wire pipes" is 0.2 (wire pipes are used to protect electric wires), the weight of "junction boxes" is 0.15 (junction boxes play an important role in circuit connection and branching), the weight of "bridges" is 0.2 (bridges are used to carry and protect large numbers of electric wires), and the weight of "grounding" is 0.2 (grounding is the key to electrical safety).

[0178] Suppose an electrician describes the process of laying electrical lines in a short sentence to be evaluated: "I first pass the wires through the conduit, and then fix the conduit to the bridge."

[0179] The server performs keyword matching on this short sentence. It contains the keywords "wire," "conduit," and "bridge." "Wire" has a weight of 0.25, a perfect match, resulting in a score of 0.25; "conduit" has a weight of 0.2, a perfect match, resulting in a score of 0.2; and "bridge" has a weight of 0.2, a perfect match, resulting in a score of 0.2. Therefore, the pending keyword score for this short sentence is 0.25 + 0.2 + 0.2 = 0.65.

[0180] In the above example of electrical engineering installation, the pending keyword score is 0.65, and the keyword score threshold is 0.3. Because 0.65>0.3, the server uses 0.65 as the final keyword score for this short sentence to be evaluated.

[0181] The server uses a suitable semantic similarity algorithm, such as the cosine similarity algorithm based on word vectors, to perform semantic matching between each short sentence to be evaluated and the standard answer text.

[0182] For example, the standard answer text is "When laying electrical lines, you must first select wires of appropriate specifications according to the design requirements, and pass the wires into pre-installed wire conduits. The conduits must be firmly fixed on the bridge, and ensure that the installation of the bridge complies with safety regulations. At the same time, pay attention to installing junction boxes where necessary to achieve circuit connection and branching. Good grounding must be ensured during the entire line laying process."

[0183] For the short sentence to be evaluated, "I first thread the wires through the conduit, then secure the conduit to the cable tray," the server calculates the semantic similarity between it and the corresponding portion of the standard answer. Through word embedding conversion and cosine similarity calculation, the proposed semantic similarity score is 0.8.

[0184] In this example of an electrical engineering installation, the pending semantic similarity score is 0.8, and the semantic matching score threshold is 0.6. Because 0.8 > 0.6, the server uses 0.8 as the final semantic similarity score for this short sentence to be evaluated.

[0185] Through this process, the server accurately calculates the keyword score and semantic similarity score for each short sentence to be evaluated based on the keywords, keyword weights, and scoring strategies configured in the test points. This helps comprehensively and objectively assess the candidate's skill level in the electrical wiring laying process, providing a reliable data foundation for the entire electrical engineering installation skills assessment. In different professional skills assessment scenarios, this process can be followed to accurately perform scoring operations based on the respective test point configuration and assessment requirements.

[0186] In an embodiment of the present invention, the construction of a sample data set for the target professional skill assessment scenario can be implemented through the following examples.

[0187] Filtering voice data and text data related to the target professional skill assessment scenario from a plurality of different professional skill assessment history records;

[0188] performing at least one data enhancement operation of adding noise, adjusting speech speed, and adjusting voice intonation on the filtered speech data;

[0189] The speech data after the data enhancement operation is combined with the corresponding text data to form the sample data set.

[0190] In the embodiment of the present invention, for example, it is assumed that the server is constructing a sample data set for an accounting professional skills assessment scenario.

[0191] The server stores a large amount of historical records of various professional skills assessments, including assessment data from different industries and positions. For the accounting professional skills assessment scenario, the server first needs to filter out relevant voice and text data.

[0192] For example, among the numerous historical assessment records, some are about engineer skill assessments, and some are about chef skill assessments. These data that are not related to accountant skill assessments will be excluded. As for the assessment records related to accounting, the server will carefully screen them.

[0193] Voice data might include recordings of accountants explaining financial statements, describing the accounting voucher processing process, and explaining tax calculations. For example, an accountant might explain how to prepare a balance sheet: "First, identify asset items such as cash, accounts receivable, and inventory. Then, identify liability items such as short-term loans and accounts payable. Finally, prepare the balance sheet according to the accounting equation."

[0194] The text data corresponds to these voice recordings, accurately recording accounting procedures, principles, calculation methods, and more. For example, "Balance sheet preparation steps: First, identify asset items, including cash, accounts receivable, and inventory; then identify liability items, including short-term loans and accounts payable; and prepare the balance sheet according to the accounting equation."

[0195] The server sifts through a vast amount of historical assessment records to extract voice and text data related to nursing care. Voice data might be a nurse describing the patient care process, such as, "When I was caring for this postoperative patient, I first checked his vital signs, including temperature, blood pressure, and heart rate. Then I checked the wound dressing for bleeding."

[0196] The corresponding text data is: "Postoperative patient care: First check vital signs (body temperature, blood pressure, heart rate, etc.), then check the wound dressing for bleeding."

[0197] For the previously filtered voice data of accountants compiling balance sheets, the server adds office ambient noise to simulate a real office scene. For example, it adds the sounds of printers printing documents and colleagues whispering to each other. This way, the originally clear voice message "First, identify asset items such as cash, accounts receivable, and inventory. Then, identify liability items such as short-term loans and accounts payable. Finally, prepare the balance sheet according to the accounting equation" becomes a voice message with a noisy office background, but the content remains unchanged.

[0198] The server can also adjust the speed of the accountant's voice data. It can speed up a normal speech to read, "First, identify asset items like cash, accounts receivable, and inventory. Then, identify liability items like short-term loans and accounts payable. Finally, prepare the balance sheet according to the accounting equation." Or it can slow down the speech to read, "First, identify asset items like cash, accounts receivable, and inventory. Then, identify liability items like short-term loans and accounts payable. Finally, prepare the balance sheet according to the accounting equation."

[0199] At the same time, adjust the tone of the voice. For example, change the tone of a plain narrative to one that emphasizes key steps. When mentioning "preparing a balance sheet according to the accounting equation," raise the tone to highlight this key step.

[0200] For the voice data of nurses describing the patient care process, the server adds noise typical of the hospital environment, such as the beeping of medical equipment and footsteps outside the ward. For example, "When caring for this postoperative patient, I first checked his vital signs, including temperature, blood pressure, and heart rate, and then checked the wound dressing for bleeding." This voice data will be accompanied by the noisy sounds of the hospital environment.

[0201] A nurse's normal speech speed can be sped up, such as "When caring for a postoperative patient, I first check his vital signs, temperature, blood pressure, heart rate, etc., and then check the wound dressing for bleeding." Or it can be slowed down to "When caring for this postoperative patient, I first check his vital signs, including temperature, blood pressure, and heart rate. Then, I check the wound dressing to see if there is any bleeding."

[0202] When referring to "check the wound dressing for bleeding," adjust your tone to a more cautious, serious one to reflect the importance of this step.

[0203] The server combines the voice data of the accountant compiling the balance sheet, which has been adjusted for noise addition, speech speed, and tone, with the corresponding text data "Balance sheet preparation steps: first determine asset items, including cash, accounts receivable, inventory, etc.; then determine liability items, including short-term loans, accounts payable, etc.; prepare the balance sheet according to the accounting equation."

[0204] Each set of augmented speech data is paired with accurate text data to form part of the sample dataset. For example, speech data about preparing a balance sheet, characterized by office noise, rapid speech, and emphasized intonation, is combined with the corresponding text data. All speech-text data pairs resulting from different augmentation methods are included in the sample dataset and used for subsequent model training, such as training a speech recognition model for accounting professional skills assessment.

[0205] Similarly, the server combines the voice data of nurses caring for patients, which has been modified with hospital environmental noise, different speech speeds, and intonations, with the corresponding text data "Postoperative patient care: first check vital signs (temperature, blood pressure, heart rate, etc.), then check for bleeding from wound dressings."

[0206] Each enhanced speech data set is paired with the corresponding text data and placed into a sample dataset. This sample dataset will provide a rich data resource for training the speech recognition model for nursing professional skills assessment, enabling the model to better adapt to different speech situations and improving the accuracy of speech recognition in nursing professional skills assessment scenarios.

[0207] In order to more clearly describe the solution provided by the embodiment of the present invention, a more detailed implementation method is provided below.

[0208] The present invention provides a voice input assessment method (device, equipment, and medium) for use in vocational skills assessment scenarios. This method utilizes speech recognition technology and natural language semantic understanding technology to address complex scenarios in vocational skills training and evaluation that are difficult to assess visually, such as the recognition of complex actions, the validity of tools, and the description of operating procedures. The implementation method of the present invention involves end-to-end speech recognition, natural language understanding, and other fields. It is divided into the following four stages:

[0209] (1) Configuration of test questions and test points: that is, for the specific test points in the training and evaluation scenarios to be identified, the candidates should verbally express the content, organize the standard text, and clearly describe the content to be tested.

[0210] (2) Constructing a dataset: Collecting voice data from different test takers in real-world assessment scenarios, as well as recording data from different people using standard text, and augmenting this data to varying degrees. This data is annotated and used as training data for the speech recognition model and the semantic understanding model.

[0211] (3) Speech recognition: SeACoParaformer, a customized non-autoregressive speech recognition model based on hot words, is adopted. On this basis, in order to improve the robustness and accuracy of the model, this solution adds a synonym processing module. Through in-depth optimization of the specific corpus for vocational skills training and evaluation scenarios, the model can more accurately capture the language characteristics of the candidates' oral expressions and convert the language signals of the candidates' oral expressions into text information.

[0212] (4) Scoring stage: In order to make the assessment results more truly reflect the actual situation of vocational skills training and evaluation, two schemes, keyword matching and semantic similarity calculation, are implemented in parallel. The purpose is to accurately assess the candidates' mastery of the key points of the operational skills assessment, and at the same time effectively prevent candidates from responding to the assessment through mechanical memorization or reciting answers, thereby ensuring the fairness, flexibility and authenticity of the evaluation.

[0213] (5) Calculate the comprehensive score: Calculate the total score based on the configured speech recognition and semantic understanding weights, thresholds and other parameters and their respective scores.

[0214] Phase 1: Test Point Configuration. This phase involves the process configuration of the test questions and test points for the assessment scenario. For the voice test points, the standard text of the answers that the candidates need to state, as well as keywords and their weights, scoring method thresholds, etc. are configured.

[0215] A test question is divided into multiple links in the process, and the links are connected in series through conditional components. Multiple business components (vision and voice) within a link can be executed in parallel and multiple times within the link.

[0216] Configure the score weights for keyword matching and semantic understanding matching for each voice test point, denoted as W key and W simu , so that W key +W simu =1.

[0217] For the keyword matching assessment option, you also need to configure the keywords that match the text of each test point, as well as their proportion in the list of keywords and synonyms of the test point.

[0218] Phase 2: Constructing the data set. To ensure the richness and balance of the data, three methods are used for data collection: first, voice data of candidates from different regions and age groups in real exam scenarios are collected; second, volunteers with different voice characteristics are invited to record voice data based on the exam intentions and standard texts, and volunteers are allowed to add appropriate personal style and understanding to simulate the diversity in real exams; third, data enhancement is performed on the data obtained by technical means, including but not limited to speech speed adjustment, volume change, noise addition, and echo simulation. After manual annotation of the above data, the training data set and the test data set are divided into a ratio of 7:3, and these data will not be used subsequently.

[0219] The third stage: speech recognition. The vocational skills assessment scenario requires scoring the real-time voice stream of the examinee. Given the diversity of the examinee group in the assessment scenario, including a wide age span, various accent characteristics, and different speaking speeds, this greatly increases the difficulty of the model covering all situations. This solution selected the SeACoParaformer with hot word speech recognition model, which has excellent performance for Chinese ASR, but encountered problems such as misrecognition of homophones and loss of speech features due to truncation during use. In the future, it will be necessary to perform tasks such as semantic understanding. Therefore, based on the original architecture, a synonym enhancement module was added:

[0220] The hotword module in SeACoParaformer captures the correlation between the Predictor output in the Paraformer and hotwords, and predicts hotword outputs that are synchronized with the ASR results. By integrating posterior probabilities, hotword motivation is achieved. This can reduce transcription errors in speech-similar situations. For example, in this scenario, "insulated shoes" is likely to be transcribed as "military shoes" if the speaker is unclear.

[0221] To improve the model's stability and reliability in different environments, with different speakers, at different speech rates and intonations, and to enhance its adaptability to speech diversity, this solution adds a synonym enhancement module to the SeACoParaformer architecture. This module primarily consists of an encoder and a decoder. Its input is a synonym dictionary and text optimized by the hotword module, and its output is a text sequence enhanced with synonyms. The specific processing process is as follows:

[0222] Synonym embedding retrieval: For each word w after word segmentation of the input text i , find its synonym set S through the synonym dictionary i . i and each synonym s in the set j ∈S i , use the pre-trained embedding table to retrieve its corresponding embedding vector e i and e sj , and calculate their respective position codes p i and p sj .

[0223] Context fusion: To select and fuse synonym embeddings, an attention mechanism is used to realize the influence of context, where the attention weight α ij It is calculated dynamically based on the context. First, the weight score is calculated:

[0224]

[0225] where d k is the dimension of the embedding vector, which is then converted into a probability distribution through the softmax function to obtain the attention weight:

[0226]

[0227] Word embedding fusion: Use the calculated attention weights to perform weighted averaging on synonym embeddings to obtain the fused synonym embedding representation:

[0228]

[0229] Enhanced vector generation: embedding the original word into e i (its weight α i ) is combined with the synonym embedding representation obtained through the above context fusion process (or called fused synonym embedding) to generate the final enhanced embedding vector e' i .

[0230] e′ i =α i ·e i +(1-α i)·fused_embed i

[0231] So far, each word w i There is a corresponding enhancement vector e' i , this vector not only contains the information of the word itself, but also integrates the semantic information of related synonyms.

[0232] When decoding, the Decoder uses the cross-attension mechanism to combine the enhanced embedding vector of the Encoder, which is used to calculate the similarity between the Decoder state and the Encoder enhanced embedding vector. Let the current state of the Decoder s t Converted into query vector q t , the cross-attention output is expressed as:

[0233]

[0234] Where K', V' are the encoder enhanced embedding vectors e' i The key and value matrix representation of .

[0235] This cross-attention output is combined with the self-attention output to generate the final decoder state representation, which contains both the self-attention information within the Decoder and the enhanced embedding vector information of the Encoder.

[0236] Vocabulary selection: Based on the current decoding state (combined with self-attention and cross-attention information), the decoder selects the most appropriate word or synonym from the vocabulary for output. This layer maps the decoder state to a score vector of the vocabulary size. The softmax function is then used to convert the score into a probability distribution and the word with the highest probability is selected as the output:

[0237] p(y t |y 1:t-1 )=softmax(Ws t +b)

[0238] Where W is the weight matrix of the output layer, b is the bias term, and s t is the current decoder state.

[0239] During real-time speech transcription, incorrect truncation is often encountered due to the complexity of the environment. For example, during periods of silence, due to high noise levels, the silence period is not detected, and forced truncation occurs after a certain duration. This causes a complete sentence to be fragmented into multiple speech segments, affecting recognition accuracy. For example, as shown in the figure below, "The insulation ladder is in good condition and has no cracks within the valid period" is a complete test point, but it is fragmented into two segments during recognition.

[0240] In Phase 1, we clearly defined a set of assessment questions as multiple sections. Furthermore, within a section, the order of test-takers' operations could be disrupted, and parallel computational processes needed to be designed to improve reasoning efficiency. To address these issues, we provided a new, customizable solution called Section Paragraphs. Its two innovative features are as follows:

[0241] Within a single step, users' responses to multiple test points are aggregated and concatenated into a single paragraph. This paragraph consists of multiple short sentences separated by punctuation marks transcribed by speech recognition to avoid semantic segmentation anomalies caused by unreasonable truncation. As shown in the figure below, multiple short sentences in a row represent segments transcribed by speech recognition, while sentences of the same color represent a logically coherent transcript of a single test point.

[0242] The previous grading scheme compared the real-time transcription of the speech with the standard text of the test site to determine the score. The new scheme reverses this process, using the standard text of the current test site to match the best answer to the segment paragraph.

[0243] The fourth stage: the scoring stage. The speech transcription text output from the third stage is accumulated in a large link to form a paragraph, and then the standard text answer of the test point is used to select a suitable sliding window win to find the most matching answer from the paragraph. The size of the sliding window is the length of the list after semantic segmentation of the answer; starting from the latest answer in the paragraph, 1 to win short sentences are spliced ​​forward and matched with the answers in turn for scoring. The specific scoring rules are as follows: First, configure the test point keyword score and the semantic matching score threshold t key and t simu , the threshold is used to determine whether the score is valid.

[0244] Starting from the latest sentence in the paragraph, loop forward and select 1 to win sentence combinations, and perform subsequent steps on the selected sentence combinations.

[0245] Keyword matching score: the keyword KW configured in the test point i , respectively match the selected phrase combination and calculate the total score of keyword matching:

[0246]

[0247] Where s is the total score of the changed test center configuration, w i is the weight of a keyword. key <t key , then s key =0, indicating that the test point failed to match the keyword and no points were awarded.

[0248] Semantic similarity matching score: Match the standard text in the test point with the selected short sentence combination to obtain the semantic matching score s simu , if s simu <t simu , then s simu =0, indicating that the test point fails to match the semantics and no points are awarded.

[0249] Phase 5: Calculate the comprehensive score. Calculate the overall score for the test point based on the configured keyword matching score and semantic understanding score. You can also configure corresponding weight coefficients for the keyword matching score and semantic understanding score respectively.

[0250] s total =s key +s simu

[0251] Please refer to Figure 2 , Figure 2 A speech input evaluation device 110 for professional skills assessment provided in an embodiment of the present invention includes:

[0252] Acquisition module 1101 is used to obtain test site configuration data for a target professional skills assessment scenario, the test site configuration data including standard answer text; construct a sample data set for the target professional skills assessment scenario, and train the enhanced SeACoParaformer architecture based on the sample data set to obtain a trained professional skills assessment speech recognition model; input the speech stream to be evaluated into the professional skills assessment speech recognition model for recognition, and obtain the text to be evaluated corresponding to the speech stream to be evaluated;

[0253] The evaluation module 1102 is used to score the text to be evaluated based on the standard answer text to obtain a keyword score and a semantic similarity score; and use the keyword score and the semantic similarity score as the professional skill evaluation score corresponding to the voice stream to be evaluated.

[0254] It should be noted that the implementation principle of the aforementioned voice input evaluation device 110 for professional skills assessment can refer to the implementation principle of the aforementioned voice input evaluation method for professional skills assessment, and will not be repeated here. It should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity, or physically separated. Moreover, these modules can be implemented entirely in the form of software called by processing elements; or entirely in the form of hardware; or some modules can be implemented in the form of processing elements calling software, and some modules can be implemented in the form of hardware. For example, the voice input evaluation device 110 for professional skills assessment can be a separately established processing element, or it can be integrated into a chip of the above device. In addition, it can be stored in the form of program code in the memory of the above device, and called by a processing element of the above device to perform the functions of the above voice input evaluation device 110 for professional skills assessment. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each module above may be completed by an integrated logic circuit of hardware in a processor element or by instructions in the form of software.

[0255] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code on a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0256] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned voice input evaluation device 110 for professional skill evaluation. Figure 3 As shown, Figure 3This is a block diagram of a computer device 100 according to an embodiment of the present invention. The computer device 100 includes a speech input evaluation device 110 for professional skill evaluation, a memory 111 , a processor 112 , and a communication unit 113 .

[0257] In order to realize the transmission or interaction of data, the memory 111, the processor 112 and the communication unit 113 are electrically connected to each other directly or indirectly. For example, the electrical connection between these elements can be achieved through one or more communication buses or signal lines. The voice input evaluation device 110 for professional skill assessment includes at least one software function module that can be stored in the memory 111 in the form of software or firmware or solidified in the operating system (OS) of the computer device 100. The processor 112 is used to execute the voice input evaluation device 110 for professional skill assessment stored in the memory 111, such as the software function modules and computer programs included in the voice input evaluation device 110 for professional skill assessment.

[0258] An embodiment of the present invention provides a readable storage medium, which includes a computer program. When the computer program is running, it controls the computer device where the readable storage medium is located to execute the aforementioned voice input evaluation device 110 for professional skills assessment.

[0259] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in light of the above teachings. These embodiments have been selected and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the present disclosure and to utilize various embodiments with various modifications as appropriate for the specific application contemplated.

Claims

1. A voice input assessment method for professional skills assessment, characterized in that: include: Obtaining test site configuration data for a target professional skills assessment scenario, wherein the test site configuration data includes standard answer text; Constructing a sample dataset for the target occupational skills assessment scenario, and training the enhanced SeACoParaformer architecture based on the sample dataset to obtain a trained occupational skills assessment speech recognition model; Inputting the speech stream to be evaluated into the professional skills assessment speech recognition model for recognition to obtain the text to be evaluated corresponding to the speech stream to be evaluated; Scoring the text to be evaluated based on the standard answer text to obtain a keyword score and a semantic similarity score; Using the keyword score and the semantic similarity score as the professional skill evaluation score corresponding to the speech stream to be evaluated; The enhanced SeACoParaformer architecture includes a hot word component and a synonym enhancement component. The input of the speech stream to be evaluated into the professional skill assessment speech recognition model for recognition to obtain the text to be evaluated corresponding to the speech stream to be evaluated includes: The speech stream to be evaluated is optimized by the hot word component to obtain a text to be processed and input into the synonym enhancement component; Performing synonym enhancement on the text to be processed based on the synonym enhancement component to obtain an enhanced text sequence; Using the text sequence as the text to be evaluated corresponding to the speech stream to be evaluated; The process of performing synonym enhancement on the text to be processed based on the synonym enhancement component to obtain an enhanced text sequence includes: Performing word segmentation processing on the text to be processed to obtain multiple word segments, and using a synonym dictionary to determine a synonym set for each word segment; using a pre-trained embedding table to retrieve the word segmentation embedding vector and the synonym set embedding vector corresponding to each word segment and each synonym set, and determining the position codes corresponding to the word segmentation embedding vector and the synonym set embedding vector, thereby realizing synonym embedding retrieval; An attention mechanism is used to combine the word segmentation embedding vector, the synonym set embedding vector, and the position encoding corresponding to each of the word segmentation embedding vector and the synonym set embedding vector to determine an attention weight to achieve context fusion; Performing weighted averaging on the synonym set embedding vectors based on the attention weights to obtain a fused synonym set embedding vector, thereby achieving word embedding fusion; Calculate an enhanced word segmentation embedding vector for each word segmentation based on the word segmentation embedding vector, the word weight pre-configured for the word segmentation embedding vector, and the synonym set embedding vector to generate an enhanced vector; Decoding the enhanced word segmentation embedding vector of each word segmentation in combination with the cross-attention mechanism to obtain a decoding state of the enhanced word segmentation embedding vector of each word segmentation, wherein the decoding state combines the self-attention mechanism and the cross-self-attention mechanism; Mapping each of the decoding states to a vocabulary to obtain a score vector corresponding to the vocabulary for each of the segmented words; transforming the score vector using a softmax function to obtain a probability distribution of each segmented word corresponding to the vocabulary, and outputting an output segmented word for each segmented word included in the text to be processed based on the probability distribution to achieve vocabulary selection; The output word segmentations are integrated to obtain the enhanced text sequence.

2. The method according to claim 1, characterized in that The test point configuration data includes a plurality of test question links, and the inputting of the to-be-evaluated voice stream into the professional skill assessment voice recognition model for recognition to obtain the to-be-evaluated text corresponding to the to-be-evaluated voice stream includes: In the target test question phase, the speech stream to be evaluated is input into the professional skill assessment speech recognition model for recognition, and at least one text to be evaluated corresponding to the speech stream to be evaluated is obtained; The at least one text to be evaluated is integrated to obtain a text to be evaluated corresponding to the target test question segment.

3. The method according to claim 2, characterized in that Scoring the text to be evaluated based on the standard answer text to obtain a keyword score and a semantic similarity score, including: Based on the standard answer text, a sliding window of the text to be evaluated corresponding to the target test question is formed; Based on the size of the sliding window, semantic segmentation is performed on the text to be evaluated corresponding to the target test question to obtain a plurality of short sentences to be evaluated; Based on the standard answer text and each of the short sentences to be evaluated, scores are performed in turn to obtain keyword scores and semantic similarity scores.

4. The method according to claim 3, characterized in that The test point configuration also includes keywords, keyword weights, and scoring strategies. The standard answer text is scored with each of the short sentences to be evaluated in turn to obtain a keyword score and a semantic similarity score, including: Obtaining a keyword scoring threshold and a semantic matching scoring threshold according to the scoring strategy; Perform keyword matching on each of the short sentences to be evaluated according to the keywords and the keyword weights to obtain a pending keyword score; In the case where the pending keyword score exceeds the keyword scoring threshold, the pending keyword score is used as the final keyword score; Perform semantic matching on each of the short sentences to be evaluated according to the standard answer text to obtain a pending semantic similarity score; In a case where the pending semantic similarity score exceeds the semantic matching score threshold, the pending semantic similarity score is used as the final semantic similarity score.

5. The method according to claim 1, wherein The constructing of a sample data set for the target occupational skills assessment scenario includes: Filtering voice data and text data related to the target professional skill assessment scenario from a plurality of different professional skill assessment history records; performing at least one data enhancement operation of adding noise, adjusting speech speed, and adjusting voice intonation on the filtered speech data; The speech data after the data enhancement operation is combined with the corresponding text data to form the sample data set.

6. A voice input assessment device for vocational skills assessment, characterized in that: include: An acquisition module is configured to obtain test site configuration data for a target vocational skill assessment scenario, the test site configuration data including standard answer text; construct a sample data set for the target vocational skill assessment scenario, and train the enhanced SeACoParaformer architecture based on the sample data set to obtain a trained vocational skill assessment speech recognition model; input a speech stream to be evaluated into the vocational skill assessment speech recognition model for recognition, and obtain the text to be evaluated corresponding to the speech stream to be evaluated; An evaluation module is configured to score the text to be evaluated based on the standard answer text to obtain a keyword score and a semantic similarity score; and use the keyword score and the semantic similarity score as a professional skill evaluation score corresponding to the speech stream to be evaluated; The enhanced SeACoParaformer architecture includes a hot word component and a synonym enhancement component. The acquisition module is specifically used to: The speech stream to be evaluated is optimized by the hot word component to obtain a text to be processed, which is input into the synonym enhancement component; the synonym enhancement component is used to perform synonym enhancement on the text to be processed to obtain an enhanced text sequence; and the text sequence is used as the text to be evaluated corresponding to the speech stream to be evaluated; The acquisition module is further specifically configured to: The text to be processed is segmented to obtain a plurality of segmented words, and a synonym dictionary is used to determine a synonym set for each segmented word; a segmented word embedding vector and a synonym set embedding vector corresponding to each segmented word and each synonym set are retrieved using a pre-trained embedding table, and the position codes corresponding to the segmented word embedding vector and the synonym set embedding vector are determined to implement synonym embedding retrieval; an attention mechanism is used to combine the segmented word embedding vector, the synonym set embedding vector, and the position codes corresponding to the segmented word embedding vector and the synonym set embedding vector to determine an attention weight to implement context fusion; the synonym set embedding vector is weighted averaged based on the attention weight to obtain a fused synonym set embedding vector to implement word embedding fusion; According to the word segmentation embedding vector, the word weight pre-configured for the word segmentation embedding vector, and the synonym set embedding vector, an enhanced word segmentation embedding vector of each word segmentation is calculated to realize enhanced vector generation; the enhanced word segmentation embedding vector of each word segmentation is decoded in combination with the cross-attention mechanism to obtain a decoding state of the enhanced word segmentation embedding vector of each word segmentation, and the decoding state is combined with the self-attention mechanism and the cross-self-attention mechanism; each decoding state is mapped to a vocabulary to obtain a score vector corresponding to the vocabulary for each word segmentation; the score vector is transformed using a softmax function to obtain a probability distribution of each word segmentation corresponding to the vocabulary, and based on the probability distribution, an output word segmentation of each word segmentation included in the text to be processed is output to realize vocabulary selection; the output word segmentations are integrated to obtain the enhanced text sequence.

7. A computer device, characterized in that: The computer device includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device executes the method according to any one of claims 1 to 5.

8. A readable storage medium, characterized in that: The readable storage medium includes a computer program, and when the computer program is executed, the computer device where the readable storage medium is located is controlled to execute the method according to any one of claims 1 to 5.

Images