Information processing method and device

Abstract

The invention discloses an information processing method and device, relates to the field of information processing, and is used for solving the problem of inaccurate verbal skill level of manual evaluation of sales personnel. The method comprises the steps that a first question item and a reference solution text are obtained, the reference solution text comprises a standard text for answering the first question item and a first scoring standard, the first scoring standard comprises a scoring point and a red line point in the standard text, the scoring point is used for indicating correct answering content, and the red line point is used for indicating wrong answering content; a first answer text input by the user is received, the first answer text is a text obtained when the user answers the first question item, the first answer text is matched with the red line point, if matching succeeds, a first score is output, if matching fails, a second score is output according to the first answer text and the score point, and the second score is higher than the first score. Through the scheme, the verbal skill level of salesmen can be accurately evaluated.

Description

Technical Field

[0001] The present invention relates to the field of information processing, and in particular to an information processing method and device. Background Art

[0002] Sales pitch training is a key part of sales and marketing training. High-quality marketing pitches directly affect the success rate of sales. Usually, companies provide sales staff with pitch templates for them to practice, and sales staff make personalized adjustments based on them to form their own marketing pitches.

[0003] The current method of evaluating sales staff's sales skills is: the training teacher distributes sales skills exercise books, the sales staff practice offline, and then uploads the practice recording to the system. The training teacher listens to the recording and completes the evaluation in the background, and the system feeds back the evaluation results to the sales staff. However, the manual evaluation method relies on manual subjective judgment, and due to inconsistent manual evaluation standards, there are large differences in evaluation.

[0004] Therefore, how to accurately evaluate the sales staff's speaking skills is an urgent problem that needs to be solved. Summary of the invention

[0005] The present invention provides an information processing method and device for accurately evaluating the salesperson's speech skills.

[0006] In a first aspect, the present invention provides an information processing method, the method comprising: obtaining a first question item and a reference answer text, the reference answer text comprising a standard text for answering the first question item and a first scoring standard, the first scoring standard comprising scoring points and red line points in the standard text, the scoring points being used to indicate correct answer content, and the red line points being used to indicate incorrect answer content, receiving a first answer text input by a user, the first answer text being the text obtained by the user answering the first question item, matching the first answer text with the red line points, outputting a first score if the match is successful, and outputting a second score if the match fails based on the first answer text and the scoring points, the second score being higher than the first score.

[0007] Through the above scheme, the first answer text can be compared with the standard text to achieve evaluation of the first answer text. Compared with the manual scoring in the prior art, the system scoring used in the present application can realize automatic scoring and improve the efficiency of scoring, and the evaluation based on the first scoring standard can unify the evaluation standard, avoiding subjective differences in human judgment; after the user enters the answer, the system can achieve real-time evaluation, support high-frequency training, and can immediately give the scoring result, which helps the user to quickly know his score and make improvements; in addition, the present application first matches the first answer text with the red line point, and then matches it with the scoring point when the match fails. If the match is successful, it means that the user has touched the red line, and then the scoring point matching is no longer performed, and the score is directly output. In this way, the number of matches between the first answer text and the scoring point can be reduced, thereby improving the efficiency of the system.

[0008] Optionally, outputting a second score based on the first answer text and the score includes: inputting the first answer text and the score into a large language model, obtaining a second score output by the large language model, and outputting the second score to a user.

[0009] Through the above scheme, automatic scoring can be achieved through the big model, and the evaluation of the first answer text input by the user is automated, reducing labor costs, and the big model can provide real-time feedback to the user's score, that is, the user can get the system's feedback evaluation in the first place. In addition, the evaluation using the big model is more efficient than the teacher evaluation in the existing technology, and the scoring standards are consistent.

[0010] Optionally, the method also includes: obtaining multiple second answer texts input by multiple users, wherein all of the multiple second answer texts fail to match the red line points in the first scoring standard; obtaining a third score for the multiple second answer texts input by professionals; inputting the multiple second answer texts and the scoring points in the first scoring standard into the large language model, and obtaining a fourth score output by the large language model, if the difference between the fourth score and the third score is greater than a set value, obtaining an adjusted first scoring standard input by the professional, repeating this step based on the adjusted first scoring standard until the difference between the fourth score and the third score is less than or equal to the set value, and using the adjusted first scoring standard to determine the scores of other answer texts for the first question.

[0011] Through the above scheme, by comparing the scores of professionals with the scores of the large language model and constantly adjusting the scoring criteria, the accuracy of the evaluation can be significantly improved. Through repeated iterations and adjustments, the scoring criteria can be continuously optimized to make it more accurate and reliable. Once an accurate scoring standard is established, the large language model can quickly process a large number of answers and improve the overall scoring efficiency.

[0012] Optionally, obtaining multiple second answer texts input by multiple users includes: obtaining N third answer texts input by multiple users, where N is an integer greater than or equal to 2, and sampling multiple second answer texts from the N third answer texts according to a set sampling rule, and the number of the multiple second answer texts is a set sampling number. In this way, using the preset sampling rules and the total number of samples, it is possible to flexibly select appropriate samples from a large number of answers, while ensuring data quality, reducing the amount of data to be processed, and improving overall efficiency.

[0013] Optionally, answer texts that meet the total number of samples are sampled from N third answer texts according to the sampling rules as multiple second answer texts, including: selecting a first number of second answer texts that meet the word count requirements, keyword requirements and screening rule requirements from the N third answer texts, selecting a second number of second answer texts that meet the similarity between the third answer text and the reference answer text greater than a first threshold from the N third answer texts, and selecting a third number of second answer texts that meet the cosine similarity between the third answer text and the reference answer text greater than a second threshold from the N third answer texts, wherein the sum of the first number, the second number and the third number is equal to the set sampling number.

[0014] Through the above scheme, combined with a variety of screening criteria, we can ensure the diversity of samples and the relevance to the reference answers, while also providing room for flexible adjustments to improve the quality and accuracy of subsequent scoring and analysis.

[0015] Optionally, the first scoring criterion also includes scoring rules, and the scoring rules include one or more of the following: text rules, regular matching rules, and prompt word rules. Matching the first answer text with the red line points includes: matching the first answer text with the red line points through text rules or regular matching rules. Inputting the first answer text and the scoring points into the large language model includes: inputting the first answer text, the scoring points, and the prompt word rules into the large language model.

[0016] Through the above scheme, multiple rules can be combined for evaluation, making the way of evaluating the first answer text more flexible. In addition, the use of text rules or regular matching rules can accurately match the user's answer with the preset key points (red line points) to improve the accuracy of the score. By inputting the answer content, scoring points and prompt word rules into the large language model, the system can perform a deeper semantic understanding, rather than just being limited to surface text matching, and can ensure the consistency of the scoring criteria, reduce the subjective differences caused by human factors, and thus achieve the unification of the evaluation criteria.

[0017] Optionally, receiving a first answer text input by a user includes: obtaining voice information input by the user and converting the voice information into first text information, segmenting the first text information, and replacing the segmented words in the first text information with a first hot word library to obtain second text information, wherein the first hot word library is a hot word library of natural language, segmenting the second text information, and replacing the segmented words in the second text information with a second hot word library to obtain the first answer text, wherein the second hot word library is a characteristic hot word library of the field to which the first question belongs.

[0018] Through the above scheme, the system can convert the user's voice input into text in real time, providing a basis for subsequent evaluation. The system can also set targeted hot word libraries according to different topics to adapt to the voice recognition needs in different scenarios. In addition, through two corrections, that is, through the hot word library and the hot word library in a specific field, the converted text can be intelligently corrected to improve the accuracy of the text and the accuracy of voice recognition.

[0019] In a second aspect, the present invention provides an information processing device, the device comprising:

[0020] an acquisition module, used to acquire a first question item and a reference answer text, wherein the reference answer text includes a standard text for answering the first question item and a first scoring standard, wherein the first scoring standard includes scoring points and red line points in the standard text, wherein the scoring points are used to indicate a correct answer content, and the red line points are used to indicate an incorrect answer content;

[0021] A receiving module, configured to receive a first answer text input by a user, wherein the first answer text is a text obtained by the user answering the first question item;

[0022] A matching module, used for matching the first answer text with the red line points;

[0023] The output module is used to output a first score if the match is successful, and to output a second score according to the first answer text and the score points if the match fails, wherein the second score is higher than the first score.

[0024] In a possible implementation, the output module is specifically used to: input the first answer text and the score point into the large language model, obtain the second score output by the large language model, and output the second score to the user.

[0025] In one possible implementation, the acquisition module is also used to: obtain multiple second answer texts input by multiple users, where all of the multiple second answer texts fail to match the red line points in the first scoring standard; obtain the third score of the multiple second answer texts input by professionals; input the multiple second answer texts and the scoring points in the first scoring standard into the large language model, and obtain a fourth score output by the large language model, if the difference between the fourth score and the third score is greater than a set value, then obtain the adjusted first scoring standard input by the professional, repeat this step based on the adjusted first scoring standard until the difference between the fourth score and the third score is less than or equal to the set value, and use the adjusted first scoring standard to determine the scores of other answer texts for the first question.

[0026] In one possible implementation, the acquisition module is specifically used to: obtain N third answer texts input by multiple users, N is an integer greater than or equal to 2, and sample multiple second answer texts from the N third answer texts according to a set sampling rule, and the number of the multiple second answer texts is the set sampling number.

[0027] In a possible implementation, the acquisition module is also used to: select a first number of second answer texts from N third answer texts that meet the word count requirement, keyword requirement and screening rule requirement, select a second number of second answer texts from the N third answer texts that meet the similarity between the third answer text and the reference answer text greater than a first threshold, and select a third number of second answer texts from the N third answer texts that meet the cosine similarity between the third answer text and the reference answer text greater than a second threshold, wherein the sum of the first number, the second number and the third number is equal to the set sampling number.

[0028] In a possible implementation, the first scoring criterion also includes a scoring rule, and the scoring rule includes one or more of the following: text rules, regular matching rules, and prompt word rules. The matching module is specifically used to: match the first answer text with the red line points through text rules or regular matching rules, and input the first answer text and the scoring points into the large language model, including: inputting the first answer text, the scoring points and the prompt word rules into the large language model.

[0029] In a possible implementation, the receiving module is specifically used to: obtain voice information input by the user, and convert the voice information into first text information, segment the first text information, and use the first hot word library to replace the segmented words in the first text information to obtain second text information, the first hot word library is a hot word library of natural language, segment the second text information, and use the second hot word library to replace the segmented words in the second text information to obtain a first answer text, and the second hot word library is a characteristic hot word library of the field to which the first question belongs.

[0030] In a third aspect, the present invention further provides an information processing device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, it implements the method described in the various possible designs of the first aspect.

[0031] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program or instructions stored thereon. When the computer program or instructions are executed by a processor, the method described in the various possible designs of the first aspect is implemented.

[0032] In a fifth aspect, the present invention further provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect above.

[0033] These implementations or other implementations of the present application will be more concise and understandable in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 A schematic diagram of a process for converting speech into readable text provided by an embodiment of the present invention;

[0036] Figure 2 A flowchart of an information processing method provided by an embodiment of the present invention;

[0037] Figure 3 A schematic diagram of setting a first question item and a reference answer text provided by an embodiment of the present invention;

[0038] Figure 4 A schematic diagram of another method of setting a first question item and a reference answer text provided in an embodiment of the present invention;

[0039] Figure 5 A schematic diagram of a process of receiving a first answer text input by a user provided by an embodiment of the present invention;

[0040] Figure 6 A schematic diagram of another information processing flow provided by an embodiment of the present invention;

[0041] Figure 7a A schematic diagram of a prompt word template provided by an embodiment of the present invention;

[0042] Figure 7b A schematic diagram of generating a return result based on a prompt word template of a big data provided by an embodiment of the present invention;

[0043] Figure 8 A schematic diagram of improving the accuracy of system evaluation through historical data provided by an embodiment of the present invention;

[0044] Fig. 9 A schematic diagram of a process flow of an optimization system based on historical data provided by an embodiment of the present invention;

[0045] Fig.10a A schematic diagram of generating a similarity value based on a large model provided by an embodiment of the present invention;

[0046] Fig.10b Another schematic diagram of generating similarity values ​​based on a large model provided by an embodiment of the present invention;

[0047] Fig.11 A schematic diagram of an information processing system provided by an embodiment of the present invention;

[0048] Fig.12 A schematic diagram of an information processing system evaluation technique provided by an embodiment of the present invention;

[0049] Fig.13 A schematic diagram of an information processing device provided by an embodiment of the present invention;

[0050] Fig.14 A schematic diagram of another information processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and beneficial effects of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] Below, some terms in this application are explained. It should be noted that these explanations are for the convenience of understanding by those skilled in the art and do not constitute a limitation on the scope of protection claimed in this application.

[0053] 1. Automatic Speech Recognition (ASR) technology.

[0054] ASR technology is a technology that converts human speech into computer-readable text in real time. Figure 1, showing a flowchart diagram of ASR technology converting speech into readable text, wherein ASR technology includes an acoustic model and a language model. The acoustic model uses a deep full-sequence convolutional neural network framework to describe the probabilistic relationship between speech signals and phonemes (the basic unit of speech). By training a large amount of speech data, the system learns the phoneme composition of speech and the transition probability between phonemes. The language model also uses a deep full-sequence convolutional neural network framework to describe the probabilistic relationship between words in the text. By training a large amount of text data, the language model can learn the composition of words and the transition probability between words. ASR technology generally involves the following steps:

[0055] 1. Obtain audio signals and preprocess speech samples (such as noise reduction, framing, etc.) to improve speech quality.

[0056] 2. Extract acoustic features (such as pitch, intensity, duration, timbre, etc.) from the preprocessed speech to represent the sound characteristics of the speech.

[0057] 3. Decode the speech through acoustic model and language model.

[0058] 4. The ASR system uses the acoustic model, language model, and previously extracted acoustic features to find the most likely word sequence, obtain recognition results, and correct the recognition results.

[0059] 5. Output the recognized text content.

[0060] 2. Correction of hot words.

[0061] Hotword correction refers to the optimization of recognition for rare or missing phrases in training data in ASR scenarios. These phrases usually include domain-specific nouns or names in the user's address book. The accurate recognition of these phrases is crucial to user experience and downstream tasks, but ASR systems trained with general data often have difficulty in completely correct recognition. To solve this problem, it is necessary to establish a hotword library to replace the misrecognized words to complete the correction work.

[0062] 3. Large language model (LLM).

[0063] LLM refers to deep learning models trained using large amounts of text data. A key technology for the effective operation of these models is the "Transformer" architecture, which introduces a mechanism called self-attention, which enables the model to focus on the relationship between parts of the input sequence to better understand the context. LLM performs well on a wide range of tasks by using a large number of parameters (possibly billions). During use, these models may be adjusted through techniques such as fine-tuning or hint engineering to adapt to specific tasks or optimize performance.

[0064] 4. Natural language processing (NLP).

[0065] NLP is a discipline that studies language and uses computer technology to analyze, understand and process natural language. It uses computers as a powerful tool for language research, conducts quantitative research on language information with the support of computers, and provides a language description method that is common to humans and computers. NLP mainly consists of two parts: natural language understanding (NLU) and natural language generation (NLG).

[0066] 5. Big model illusion.

[0067] The seemingly reasonable but erroneous answers provided by large language models are called "hallucinations", or in short, "nonsense", that is, the content generated by the model is inconsistent with real-world facts or user input. The main reasons for large models to produce hallucinations are data defects, low utilization of factual knowledge captured in the data, insufficient recall of long-tail knowledge, and difficulty in dealing with complex reasoning.

[0068] 6. Prompt word engineering.

[0069] Prompt word engineering is a method for optimizing large language models. It guides the model to generate more accurate and targeted outputs by carefully designing and adjusting input prompts. Its characteristics include precise control, strong scalability and practicality, but interactive learning requires professional knowledge. With the development of large-scale language models, prompt word engineering has become an important field, opening up broad possibilities for various applications.

[0070] 7. Scorecard model.

[0071] The scoring card model, also known as the credit scoring card model, was first introduced by the American credit scoring giant FICO in the 1960s and is widely used in the fields of credit risk assessment and financial risk control. Banks use the scoring card model to score multiple features of a customer's credit history data to obtain different levels of credit scores, thereby judging the quality of the customer and deciding whether to grant credit and the amount and interest rate of the credit. Compared with the expert rules set by senior practitioners based on their own experience, the use of the scoring card model has obvious advantages.

[0072] 8. Term frequency-inverse document frequency (TF-IDF).

[0073] TF-IDF is a commonly used weighting technology for information retrieval and text mining. It can be used to evaluate the importance of a word in an article and is often used in applications such as text classification and search engine ranking.

[0074] 9. Cosine Similarity

[0075] Cosine similarity is a measure of the angle between two non-zero vectors, and is often used in text analysis, information retrieval, recommendation systems, etc. Cosine similarity focuses on the direction of two vectors rather than their magnitude, so it can capture the similarity between vectors well without being affected by the length of the vectors.

[0076] As described in the background technology, accurately evaluating the salesperson's sales skills and giving corresponding evaluations can improve the salesperson's sales skills. The current method of evaluating sales skills by training teachers has many disadvantages. For example, the training process requires the participation of training teachers, the process is cumbersome, and sales personnel cannot complete it independently; and sales personnel cannot get real-time feedback after practice, and training teachers need to listen to the recording completely before evaluating, which is inefficient. Therefore, how to accurately evaluate the salesperson's sales skills is a problem that needs to be solved urgently.

[0077] Based on this, the present invention provides an information processing method for accurately evaluating the salesperson's speech skills by matching the first answer text with the red line points and the score points.

[0078] The present application scheme is described in detail below in conjunction with the accompanying drawings.

[0079] See also Figure 2 , a flow chart of an information processing method is shown, which specifically includes the following steps:

[0080] Step 210: Obtain the first question item and reference answer text.

[0081] Among them, the reference answer text includes the standard text for answering the first question and the first scoring standard. The first scoring standard includes scoring points and red line points in the standard text. The scoring points are used to indicate the correct answer content, and the red line points are used to indicate the incorrect answer content.

[0082] Optionally, the first question item and the reference answer text can be manually set, for example, by a professional (such as a training teacher) setting the first question item and the reference answer text in advance, see Figure 3 , showing a schematic diagram of setting the first question item and the reference answer text, professionals (such as training teachers) first create a test question special item, then create test questions based on the test question special item, and then create a score card and feature hot word library for the test questions, set scoring rules based on the score card, and set hot words based on the feature hot word library.

[0083] Optionally, the professional setting the first question item and the reference answer text may further include the following steps:

[0084] Step a: Professionals set the first question item and the standard text corresponding to the first question item, wherein the standard text is a reference answer to the first question item.

[0085] Step b: Professionals set up a score card based on the first question, extract scoring points from the standard text, and set corresponding scoring rules and scores for each scoring point.

[0086] Among them, the scoring points include scoring points and red line points, and the scoring rules may include one or more of the following: text rules, regular matching rules, and prompt word rules. Among them, the text rule is a rule for identifying or matching based on preset keywords or phrases, the regular matching rule is a rule for using regular expressions to match text, and the prompt word rule is a rule for generating prompt words that the first model can understand based on a preset template, and the first model can output a score based on the prompt word.

[0087] Optionally, you can set the score corresponding to the red line point to 0. The user's final score is the sum of the scores of all the hit points.

[0088] For example, the standard text corresponding to the first question item has 3 scoring points, including 1 red line point A and 2 scoring points B and C. The text rule and score 0 are set for the red line point A, the regular matching rule and score 1 are set for the scoring point B, and the prompt word rule and score 2 are set for the scoring point C. If the user's first answer text meets the red line point A according to the text rule, the user's final score is 0; if the user's first answer text does not meet the red line point A according to the text rule, meets the scoring point B according to the regular matching rule, and meets the scoring point C according to the prompt word rule, the user's final score is 1+2=3.

[0089] Optionally, the method may further include step c: a professional (such as a training teacher) sets a characteristic hot word library according to the first question item and the standard text, wherein the characteristic hot word library is related to the field to which the first question item belongs and is used to correct the text input by the user.

[0090] See also Figure 4, showing a schematic diagram of setting the first question item and the reference answer text, wherein the professional (such as a training teacher) sets the training question 1 and the training question 2 according to the test question in the above step a; in the above step b, respectively sets the scoring point a and the corresponding scoring rule a, the scoring point b and the corresponding scoring rule b corresponding to the training question 1, and sets the scoring point c and the corresponding scoring rule c, the scoring point d and the corresponding scoring rule d corresponding to the training question 2; in the above step c, sets the hot word library corresponding to the training question 1 and the hot word library corresponding to the training question 2. In this way, the setting of the first question item and the reference answer text can be completed.

[0091] Step 220: Receive a first answer text input by the user.

[0092] The first answer text is the text obtained by the user answering the first question. If the user inputs voice information, the voice information can be converted into text information.

[0093] However, if the user's Mandarin is not standard or the pronunciation is similar, it may cause speech recognition errors, such as "to be repaid" and "loan principal", "qualification" and "posture", "settlement" and "dissolution". Therefore, it is necessary to correct the misrecognized speech through the hot word library, that is, based on the initial text generated by ASR, text correction is performed to make the final text as close to the user's actual expression as possible.

[0094] Optional, see Figure 5 , showing a flow chart of receiving a first answer text input by a user. Step 220 may specifically include the following steps 510 - 580.

[0095] Step 510: Acquire voice information input by the user.

[0096] Optionally, the user can enter the practice page and click the "Start Answering" button. The system can establish a long connection with the server through the WebSocket protocol, and the server receives the voice information input by the user.

[0097] Step 520: Convert the speech information into first text information according to the speech recognition model.

[0098] The user's voice information is input into the real-time voice transcription module in the form of real-time voice stream. The module uses ASR technology to transcribe the voice stream into the initial text. For details, please refer to the above-mentioned glossary and Figure 1 .

[0099] Optionally, the voice information can be converted into text information through a public model or a self-developed model trained with a public model and labeled audio of own data.

[0100] Step 530: Segment the first text information.

[0101] Step 540: By scanning the first hot word library, determine whether there are hot words that need to be corrected in the first text information. If yes, execute step 550; if not, execute step 560.

[0102] The first hot word library is a hot word library of natural language, that is, a general hot word library.

[0103] Step 550: Correct the first text information and execute step 560.

[0104] After the first text information is segmented, the segmented data is filtered through a common hot word library, and incorrect words are replaced, so that the first text information can be corrected.

[0105] Step 560: By scanning the second hot word library, determine whether there are hot words that need to be corrected in the first text information. If yes, execute step 570; if not, execute step 580.

[0106] The second hot word library is a characteristic hot word library of the field to which the first question belongs.

[0107] Step 570: Correct the first text message again.

[0108] The first text information can be corrected twice by passing the first text information through the characteristic hot word library corresponding to the first question item and performing a second replacement on the wrong words.

[0109] Step 580: Use the final text as the first answer text.

[0110] For example, the following is an example of receiving a user's input voice information and correcting the voice information to obtain a first answer text:

[0111] First question: The customer asks what is the maximum amount they can get for using your product?

[0112] Standard text: Boss, the specific amount we can open depends on the capital.

[0113] First answer text: Boss, the amount of our pending loan product can provide mainly depends on your attitude (qualifications).

[0114] In the above process, due to personal pronunciation problems, the accents of words such as "loan" and "to be repaid", "individual" and "follow", "qualification" and "posture" are similar, which may cause the translated text to not convey the intended meaning. Therefore, the voice information input by the user can be corrected in the following two steps: Step 1: Through the general hot word library, set "to be repaid products" as a hot word, and uniformly correct "to be repaid products" to "loan products". Step 2: For the word "qualification", considering that it is used frequently in specific scenarios, and "posture" is rarely used in the answer to this question, a targeted hot word library can be set for the training question, and "posture" can be corrected to "qualification". Through two corrections, the voice information input by the user can be restored to the user's original intention.

[0115] Through the above solution, the system can convert the user's voice input into text in real time, providing a basis for subsequent evaluation. The system can also set targeted hot word libraries according to different topics to adapt to the voice recognition needs in different scenarios. In addition, through two corrections, namely, through the hot word library and the hot word library in a specific field, the converted text can be intelligently corrected to improve the accuracy of the text and the accuracy of voice recognition.

[0116] Step 230: Match the first answer text with the red line point to determine whether the match is successful. If the match is successful, execute step 240; if the match fails, execute step 250.

[0117] Optionally, matching the first answer text with the red line points includes: matching the first answer text with the red line points by text rules or regular matching rules. In this way, the user's answer can be accurately matched with the preset key points (red line points) by using text rules or regular matching rules, thereby improving the accuracy of the scoring.

[0118] This application first matches the first answer text with the red line point to determine whether the first answer text touches the red line point. If it touches the red line point, the score can be directly output without matching the first answer text with the score point.

[0119] Step 240: Output the first score.

[0120] Optionally, the first score may be 0, that is, if the user's first answer text touches the red line point, the user directly gets 0 points.

[0121] Step 250: Output a second score according to the first answer text and the score points.

[0122] If the first answer text does not touch the red line point, continue to determine how many scoring points the first answer text contains, and output the second score based on the first answer text and the scoring points. It can be understood that the second score is higher than the first score.

[0123] Optionally, outputting a second score based on the first answer text and the score point includes: inputting the first answer text and the score point into the large language model, obtaining the second score output by the large language model, and outputting the second score to the user. Through the above scheme, automatic scoring can be achieved through the large model, and the evaluation of the first answer text input by the user is automated, reducing labor costs, and the large model can provide real-time feedback to the user on the score, that is, the user can get the feedback evaluation of the system in the first place. In addition, the evaluation using the large model is more efficient than the teacher evaluation in the prior art, and the scoring standard is consistent.

[0124] Optionally, inputting the first answer text and the scoring points into the large language model includes: inputting the first answer text, the scoring points and the prompt word rules into the large language model. In this way, by inputting the answer content, the scoring points and the prompt word rules into the large language model, the system can perform a deeper level of semantic understanding, rather than just being limited to surface text matching, and can ensure the consistency of the scoring criteria, reduce the subjective differences caused by human factors, and thus achieve the unification of the evaluation criteria.

[0125] Optionally, the present application may construct a prompt word template in advance, fill the first answer text and the scoring point into the prompt word template, input the filled prompt word template into the large language model, and the large language model may output the second score. The prompt word template may include the following three items: the first question item, the first answer text, and the scoring rule (or scoring point), and may also include other contents without limitation.

[0126] Optionally, predefined templates and rules can be used to automatically generate prompt words suitable for large-model understanding and processing according to different test questions, thereby realizing automatic construction of prompt word templates.

[0127] Optionally, the large language model can be a public large model or a self-owned large model, such as Baichuan, Qianwen, ChatGPT, Doubao pre-trained large model, etc., without restriction.

[0128] Optional, see Figure 6 , showing another information processing flow diagram, including the following steps:

[0129] Step 610: Obtain the first answer text.

[0130] Step 620: Query the first question item and the reference answer text corresponding to the first answer text, and obtain the first scoring criterion in the reference answer text, and the scoring rule corresponding to the first scoring criterion.

[0131] Optionally, the first scoring criterion also includes scoring rules, and the scoring rules include one or more of the following: text rules, regular matching rules, and prompt word rules. In this way, multiple rules can be combined for evaluation, making the way of evaluating the first answer text more flexible.

[0132] Step 630: Execute the scoring rule to determine whether the first answer text matches the red line point. If so, execute step 640; if not, execute step 650.

[0133] Step 640: Prompt the user to touch the red line and execute step 680.

[0134] The first answer text is matched with the red line point. If the match is successful, it means that the user has touched the red line. A prompt message is sent to the user and step 680 is executed to output 0 points.

[0135] Step 650: Use the large model to construct prompt words.

[0136] By filling the first answer text and scoring points into the prompt word template, a large model prompt word can be constructed.

[0137] Step 660: Obtain conclusion information returned by the large model, and parse the conclusion information to obtain score points that match the first answer text.

[0138] Step 670: Obtain the scores of the score points that match the first answer text, and add them up to obtain a final score.

[0139] Get the formatted data returned by the big model. By parsing the hit points and accumulating the corresponding scores, you can get the score given by the big model.

[0140] Step 680: Output the final score.

[0141] Through the above steps 610 to 680, when the first answer text does not match the red line point, the large model is used to match the first answer text with the score point, and the user's final score is obtained by parsing the return information of the large model, thereby realizing the automation of user scoring.

[0142] The following is a specific example to explain how to use the large model for scoring. Assume that the first question item and the standard text corresponding to the first question item have been set in advance.

[0143] Item 1: You are recommending a loan product, and the customer asks you if you are a bank employee. How should you answer?

[0144] Standard text: I am not a bank employee, but the products I recommend are loan products from regular licensed financial institutions, so please rest assured.

[0145] To solve this problem, you can first set up a set of scoring cards and corresponding scoring rules. The example is as follows:

[0146] Point 1: We are not bank staff, score: 40 points

[0147] Rule type: prompt word rule.

[0148] Rule content: The answer must include the meaning that I am not a bank employee, and especially I cannot imply that I am a bank employee by recommending products.

[0149] Point 2: We recommend loan products from formal licensed financial institutions, score: 60 points

[0150] Rule type: prompt word rule.

[0151] Rule content: The answer must include a description of the loan product, and the description must contain legal and compliant information in terms of both literal meaning and semantics.

[0152] Red line points: Red line words such as "loan sharking", "lack of money", "intermediary" etc. cannot appear.

[0153] Rule type: Text rule.

[0154] Rules content: loan sharking, lack of money, and intermediaries.

[0155] Assume there are two sales answers, where:

[0156] Sales answer 1: I’m in charge of loan processing. You can come to me if you need money.

[0157] Scoring process: This answer matches the rule text: There is a rule word for lack of money, so it is directly given 0 points.

[0158] Sales answer 2: We are employees of a certain company, and we are responsible for assisting the bank in promoting loan products.

[0159] Scoring process: The answer does not hit the red line words, and then it is scored according to the scoring rules of the big model, that is, through the prompt word engineering and combined with the prompt word template, the prompt words that the big model can understand are constructed. Figure 7a A schematic diagram showing a prompt word template is shown. Figure 7a The content is sent to the big model, and the return result of the big model can be obtained, such as Figure 7b As shown, through the return value of the large model, we can know that the answer hits two scoring points, scoring point 1 (40 points) and scoring point 2 (60 points). Therefore, through the accumulation of scoring point values, we can get a score of 100 for this exercise.

[0160] Finally, through data processing of the evaluation component, the following content is returned to the user: Your answer contains two key points: 1. We are not bank employees 2. We recommend loan products from formal licensed financial institutions. At the same time, there is no illegal information in the script. Therefore, the overall evaluation of this exercise is 100 points. Please keep up the good work.

[0161] The data in this application is not only used for statistics and analysis, but also for improving the accuracy of system evaluation. Since the answers to oral expressions are diverse, the scoring rules need to be continuously optimized based on historical data, so that the evaluation accuracy can be gradually improved and eventually the system evaluation accuracy reaches the expected value.

[0162] Optionally, past data can be sampled manually and algorithmically, and manually evaluated to form a validation data set that is considered 100% accurate. Then, the scoring points and scoring rules of the test questions can be optimized based on this data. Then, the validation data set is evaluated in real time using the optimized rules, and the results are compared with the original manual evaluation to obtain the optimized evaluation accuracy.

[0163] like Figure 8 As shown in the schematic diagram of improving the accuracy of system evaluation through historical data, the user answers the first question and obtains a first answer text. The first answer text obtains two evaluation scores through manual evaluation and system evaluation respectively. Specifically, the system evaluation is to input the first answer text into the large model semantic evaluation module, and the large model semantic module determines whether the first answer text hits the three scoring points a, b, and c. The manual evaluation is to send the first answer text to the training teacher, and the training teacher determines whether the first answer text hits the three scoring points a, b, and c. Finally, the manual results are compared with the system results. If the comparison results are consistent, it proves that the system evaluation is accurate.

[0164] Optionally, after step 250, the scoring criteria may be optimized based on historical data. Fig. 9 , showing a schematic diagram of the process of optimizing the system based on historical data, which specifically includes the following steps:

[0165] Step 910: Obtain N third answer texts input by multiple users.

[0166] Wherein, N is an integer greater than or equal to 2. The third answer text may be historical data or data reacquired by the system, without limitation.

[0167] Step 920: Obtain the preset sampling rules and total number of samples.

[0168] The sampling rules may include one or more of the following: basic rules, large model similarity rules, and cosine similarity rules. In addition, the sampling ratio and sampling threshold can be set. For example, sampling can be set according to 50% of the basic rules and 50% of the cosine similarity rules. In this way, using the preset sampling rules and sampling quantity, you can flexibly select appropriate samples from a large number of solutions, while ensuring data quality, reducing the amount of data to be processed and improving overall efficiency.

[0169] Optionally, the basic rules include one or more of the following: word count requirement, keyword requirement, exercise population requirement, and screening rule requirement, wherein the screening rule requirement includes standard answers and fixed answers. Since some users have participated in the assessment many times and have memorized the standard answers or have their own fixed answers, the answer texts that are completely consistent with the standard answers and fixed answers can also be filtered out, thereby achieving preliminary filtering and sampling of historical data.

[0170] The big model similarity rule is a rule that calculates the similarity between the first answer text and the standard text based on the big model. The prompt word project can generate the prompt words for the big model similarity. The prompt words are input into the big model to obtain the similarity value between the answer text and the standard text, calculate the absolute value of the difference between the similarity value and the preset value, and determine whether the absolute value of the difference is greater than the set threshold. If it is, the answer text is used as sample data. Please refer to Fig.10a and Fig.10b Schematic diagram of generating similarity values ​​based on a large model, wherein: Fig.10a is the prompt word for the similarity of the large model, Fig.10b It is the result information returned by the large model based on the prompt word, where the result information includes a similarity value.

[0171] For example, the total number of samples is set to 300, the sampling ratio of the large model similarity rule is 50%, the threshold is 30 points, and assuming that the preset large model similarity value is 50, the answer text data with a sampling similarity value below 20 points and above 80 points are used as quality inspection sample data.

[0172] The cosine similarity rule is a rule for calculating the cosine similarity between the first answer text and the standard text. For example, by calculating the TF-IDF vector of the first answer text and the TF-IDF vector of the standard text, and calculating the cosine similarity of the two vectors, it can be determined whether the cosine similarity is less than a preset threshold. If it is less than, the answer text data is used as sampling data. For example, assuming that the preset threshold is 0.6, the answer text data with a cosine similarity lower than 0.6 is used as sampling data.

[0173] Optionally, the value range of cosine similarity is between [-1, 1]. The calculated value of cosine similarity can be added by 1 and then divided by 2 to keep the value range between [0, 1]. That is, completely similar is 1, and completely opposite is 0.

[0174] Among them, calculating the TF-IDF value of the current standard text includes the following steps:

[0175] Step A: Segment all reference answers into words and calculate the first probability of each word appearing in all reference answers.

[0176] Step B: Segment the current standard text into words and calculate the second probability of each word appearing in the current standard text.

[0177] Step C: Multiply the first probability by the second probability to obtain the TF-IDF value of the current standard text.

[0178] The same method can be used to calculate the TF-IDF value of the first answer text, and then calculate the cosine similarity between the TF-IDF of the standard text and the TF-IDF of the first answer text.

[0179] Step 930: Sample answer texts that meet the total number of samples from the N third answer texts according to the sampling rule as multiple second answer texts.

[0180] Optionally, step 930 includes: selecting a first number of second answer texts from N third answer texts that meet word count requirements, keyword requirements, and screening rule requirements, selecting a second number of second answer texts from the N third answer texts that meet the similarity between the third answer text and the reference answer text greater than a first threshold, and selecting a third number of second answer texts from the N third answer texts that meet the cosine similarity between the third answer text and the reference answer text greater than a second threshold, wherein the sum of the first number, the second number, and the third number is equal to the set sampling number.

[0181] Through the above scheme, combined with a variety of screening criteria, we can ensure the diversity of samples and the relevance to the reference answers, while also providing room for flexible adjustments to improve the quality and accuracy of subsequent scoring and analysis.

[0182] Step 940: Obtain third scores of the plurality of second answer texts input by the professionals.

[0183] Professionals can manually annotate the sample data with the hit points and form a verification data set.

[0184] Optionally, professionals can synchronize the scoring rules for the item as a copy of the quality inspection rules.

[0185] Step 950: Obtain the fourth score output by the large model.

[0186] Step 960: Determine whether the difference between the fourth score and the third score is greater than a set value, if so, execute step 970, if not, execute step 980.

[0187] Step 970: Update the current scoring criteria.

[0188] Step 980 : Obtain the second scoring standard input by the professional, and the third scores of the plurality of second answer texts obtained by the professional according to the second scoring standard, and execute step 950 .

[0189] When the difference between the fourth score and the third score is greater than the preset value, it means that the system score is not accurate enough, and professionals can optimize the scoring rule to obtain a second scoring rule. The second scoring standard includes scoring points and red line points, and for details, reference can be made to the first scoring standard mentioned above.

[0190] After that, you can continue to create quality inspection tasks and set quality inspection goals, re-evaluate the answer text in the verification data set and the optimized scoring rules through the big model semantic evaluation module to obtain the evaluation score, that is, repeat steps 950-980, compare the fourth score re-evaluated by big data with the manually annotated data in the verification data set, and obtain the system evaluation accuracy. If the evaluation accuracy meets expectations (such as an accuracy of 90%), the optimized second scoring rule will replace the first scoring rule. If it does not meet expectations, continue to optimize, and repeat the cycle until it meets expectations.

[0191] Through the above solution, professionals only need to participate in maintaining the question bank and the corresponding scoring standards, and improve the evaluation accuracy based on historical data. Compared with the existing technology, the practice evaluation of this application can be completed by the system in real time, and can cope with large-scale, high-frequency speech training scenarios with high efficiency.

[0192] The above steps 910 to 980 introduce a method for optimizing a system based on historical data. The following explains the above steps 910 to 980 using a specific example.

[0193] Assume that the first question item and standard text are set in advance as follows:

[0194] Item 1: You are recommending a loan product, and the customer asks you if you are a bank employee. How should you answer?

[0195] Standard text: I am not a bank employee, but the products I recommend are loan products from regular licensed financial institutions, so please rest assured.

[0196] The first step is to sample historical data using a sampling algorithm, and then manually annotate and evaluate them. The following are two examples:

[0197] Answer 1: Manual evaluation hits evaluation point 1 and scoring point 2.

[0198] Answer 2: Manual evaluation hits evaluation point 1.

[0199] In the second step, the first scoring standard is synchronized to the quality inspection system, and the first scoring standard is iteratively modified and evaluated again through the real-time evaluation module, and the following results are obtained:

[0200] Answer 1: The secondary evaluation of the module hits evaluation point 1 and scoring point 2.

[0201] Answer 2: The secondary evaluation of the module hits evaluation point 1.

[0202] Step 3: Compare the two evaluation results. In answer 1 and answer 2, the manual evaluation hit points are exactly the same as the evaluation module evaluation hit points. Therefore, the accuracy of the evaluation module reaches 100%, which can completely replace manual intelligent evaluation.

[0203] In the fourth step, the re-optimized first scoring criterion will replace the original first scoring criterion.

[0204] Through the above steps 910 to 980, by comparing the scores of professionals with the scores of the large language model and continuously adjusting the scoring criteria, the accuracy of the evaluation can be significantly improved, and through repeated iterations and adjustments, the scoring criteria can be continuously optimized to make it more accurate and reliable. Once an accurate scoring standard is established, the large language model can quickly process a large number of answers and improve the overall scoring efficiency.

[0205] The above content introduces the information processing method for evaluating the user's first answer text. The following uses the evaluation of sales talk as an example to explain the above method with a complete example.

[0206] First, we introduce an information processing system, see Fig.11 The system includes a real-time speech transcription module, a large model semantic evaluation module and a result feedback module. First, the salesperson inputs the speech into the real-time speech transcription module, which transcribes and corrects the sales speech to obtain text information, and then inputs the text information into the large model semantic evaluation module to obtain the conclusion of the large model output, and finally feeds back the evaluation result to the user. The system also includes a question bank management module and a data quality inspection module, wherein the question bank management module is used to collect and store questions, and the data quality inspection module is used to optimize the scoring rules based on the historical data of sales speech training and improve the accuracy of real-time evaluation.

[0207] Based on the above information processing system, the system can evaluate sales tactics, see Fig.12, showing a schematic diagram of the information processing system evaluation speech, which specifically includes the following steps:

[0208] Step 1: Construct the test questions. This includes the following 4 steps:

[0209] Step 1: Train teachers to complete the question-setting work, which includes practice questions and corresponding reference answers.

[0210] Step 2: The training teacher configures the scoring criteria for the practice questions, extracts scoring points from the reference answers, where the scoring points include scoring points and red line points, and then sets corresponding scoring rules for each scoring point. For details, please refer to the above step 210.

[0211] Step 3: Set up a targeted feature hot word library, make targeted corrections to the sales input transcription text, and finally complete the construction of a test question evaluation model.

[0212] Step 4: Use the TF-IDF semantic similarity calculation algorithm to calculate the TF-IDF value of the reference answer and record the corresponding TF-IDF vector value as the basis for subsequent data quality inspection sampling. The TF-IDF value calculation can refer to the above steps A to C.

[0213] Step 2: Get the sales practice text (i.e. the content to be evaluated). This includes the following 3 steps.

[0214] Step 1: The salesperson enters the practice page and clicks the Start Answering button. A long connection is established with the server through the WebSocket protocol. The salesperson's answers are input into the real-time voice transcription module through the real-time voice stream, and the voice stream is transcribed into initial text through ASR technology.

[0215] Step 2: Segment the initial text, filter the segmented data with a common hot word library, replace incorrect words, and form a secondary text.

[0216] Step 3: Filter the secondary text again through the characteristic hot word library corresponding to the sales practice questions, replace the wrong words for a second time, and get the final translated text.

[0217] Step 3: Real-time semantic evaluation. This includes the following 5 steps:

[0218] Step 1: Get the red line points corresponding to the scoring criteria of the test questions, match the illegal words in the red line points, and directly output 0 points if the match is successful.

[0219] Step 2: Get the scoring points corresponding to the scoring criteria of the test questions and build a large model prompt word.

[0220] Step 3: Get the formatted data returned by the big model, parse the hit scoring points, and get the big model score by accumulating the scores corresponding to the scoring points.

[0221] Step 4: Use the TF-IDF semantic similarity calculation algorithm to calculate the TF-IDF vector for the sales answer and record the data.

[0222] Step 5: Build the response and provide the score back to sales.

[0223] Step 4: Continuously optimize the system's evaluation accuracy through the data quality inspection module. This includes the following 10 steps:

[0224] Step 1: Set data sampling rules, sampling ratio, sampling threshold and total number of samples. The sampling rules include basic rules, large model similarity rules and cosine similarity rules.

[0225] Step 2: Set the number of words, keywords, practice groups, etc. in the basic rules, and set filtering rules: completely consistent with the standard answer or fixed answer, and conduct preliminary filtering sampling of historical sales answers.

[0226] Step 3: Extract sales answer data that meets the basic rules, generate large model similarity prompt words through prompt word engineering, generate similarity scores between sales answers and reference answers through prompt words, calculate the absolute value of the difference in real-time scores, detect whether the absolute value of the difference is greater than the set threshold, and use data greater than the threshold as sample data.

[0227] Step 4: Calculate the TF-IDF vector of the sales answer and the TF-IDF vector of the standard answer, calculate the similarity of the two TF-IDF vectors through cosine similarity, determine whether the similarity score is less than the threshold, and use the data less than the threshold as sample data.

[0228] Step 5: Quality inspectors manually mark the sample data and identify the scoring points to form a verification data set.

[0229] Step 6: The quality inspector synchronizes the online scoring rule data of the test question as a copy of the quality inspection rules.

[0230] Step 7: Quality inspectors optimize the scoring rule copy.

[0231] Step 8: Create a quality inspection task, set the quality inspection goal, and re-evaluate the sales answers in the verification data set and the optimized scoring rules through the large model semantic evaluation module to obtain the evaluation score.

[0232] Step 9: Compare the re-evaluation results with the manually labeled data in the validation dataset to obtain the system evaluation accuracy.

[0233] Step 10: If the accuracy meets business expectations, the optimized scoring rules will replace the scoring rules of the production environment and take effect in real time. If it does not meet expectations, continue to optimize, and repeat the cycle until it meets expectations.

[0234] Through the above scheme, the first answer text can be compared with the standard text to achieve evaluation of the first answer text. Compared with the manual scoring in the prior art, the system scoring used in the present application can realize automatic scoring and improve the efficiency of scoring, and the evaluation based on the first scoring standard can unify the evaluation standard, avoiding subjective differences in human judgment; after the user enters the answer, the system can achieve real-time evaluation, support high-frequency training, and can immediately give the scoring result, which helps the user to quickly know his score and make improvements; in addition, the present application first matches the first answer text with the red line point, and then matches it with the scoring point when the match fails. If the match is successful, it means that the user has touched the red line, and then the scoring point matching is no longer performed, and the score is directly output. In this way, the number of matches between the first answer text and the scoring point can be reduced, thereby improving the efficiency of the system.

[0235] Based on the same concept, an embodiment of the present application also provides an information processing device, which can execute the information processing method introduced in the above content.

[0236] See also Fig.13 , a schematic diagram of the structure of an information processing device provided in an embodiment of the present application is given, such as Fig.13 As shown, the information processing device includes:

[0237] An acquisition module 1301 is used to acquire a first question item and a reference answer text, wherein the reference answer text includes a standard text for answering the first question item and a first scoring standard, wherein the first scoring standard includes scoring points and red line points in the standard text, wherein the scoring points are used to indicate a correct answer content, and the red line points are used to indicate an incorrect answer content;

[0238] A receiving module 1302 is used to receive a first answer text input by a user, where the first answer text is the text obtained by the user answering the first question item;

[0239] A matching module 1303, used for matching the first answer text with the red line point;

[0240] The output module 1304 is used to output a first score if the match is successful, and to output a second score based on the first answer text and the score points if the match fails, wherein the second score is higher than the first score.

[0241] In a possible implementation, the output module 1304 is specifically configured to: input the first answer text and the score point into the large language model, obtain the second score output by the large language model, and output the second score to the user.

[0242] In a possible implementation, the acquisition module 1301 is also used to: obtain multiple second answer texts input by multiple users, where the multiple second answer texts all fail to match the red line points in the first scoring standard; obtain the third score of the multiple second answer texts input by professionals; input the multiple second answer texts and the scoring points in the first scoring standard into the large language model, and obtain the fourth score output by the large language model, if the difference between the fourth score and the third score is greater than the set value, then obtain the adjusted first scoring standard input by the professional, repeat this step based on the adjusted first scoring standard until the difference between the fourth score and the third score is less than or equal to the set value, and use the adjusted first scoring standard to determine the scores of other answer texts for the first question.

[0243] In one possible implementation, the acquisition module 1301 is specifically used to: obtain N third answer texts input by multiple users, N is an integer greater than or equal to 2, and sample multiple second answer texts from the N third answer texts according to a set sampling rule, and the number of the multiple second answer texts is the set sampling number.

[0244] In one possible implementation, the acquisition module 1301 is also used to: select a first number of second answer texts that meet the word count requirement, keyword requirement and screening rule requirement from N third answer texts, select a second number of second answer texts that meet the similarity between the third answer text and the reference answer text greater than a first threshold from the N third answer texts, and select a third number of second answer texts that meet the cosine similarity between the third answer text and the reference answer text greater than a second threshold from the N third answer texts, wherein the sum of the first number, the second number and the third number is equal to the set sampling number.

[0245] In a possible implementation, the first scoring criterion also includes a scoring rule, and the scoring rule includes one or more of the following: text rules, regular matching rules, and prompt word rules. The matching module is specifically used to: match the first answer text with the red line points through text rules or regular matching rules, and input the first answer text and the scoring points into the large language model, including: inputting the first answer text, the scoring points and the prompt word rules into the large language model.

[0246] In a possible implementation, the receiving module 1302 is specifically used to: obtain voice information input by the user, and convert the voice information into first text information, segment the first text information, and use the first hot word library to replace the segmented words in the first text information to obtain second text information, the first hot word library is a hot word library of natural language, segment the second text information, and use the second hot word library to replace the segmented words in the second text information to obtain a first answer text, and the second hot word library is a characteristic hot word library of the field to which the first question belongs.

[0247] See also Fig.14 , showing a schematic diagram of the structure of another information processing device provided in an embodiment of the present application, such as Fig.14 As shown, the test device includes: a memory 1401 and a processor 1402, and the processor 1402 is coupled to the memory 1401. The memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions stored in the memory 1401 and execute the above information processing method according to the obtained program.

[0248] Optionally, the data processing device may further include an interface circuit 1403, which may be a transceiver or an input / output interface. The input / output interface is used to input and / or output information, where output may be understood as sending and input may be understood as receiving. The processor 1402 may communicate with other devices in the test device or other devices other than the information processing device through the interface circuit 1403 to obtain information required for executing the above information processing method.

[0249] When the information processing device 1400 is used to implement Figure 2 When the method is shown, the processor 1402 is used to implement the functions of the above-mentioned acquisition module 1301, receiving module 1302, matching module 1303, and output module 1304.

[0250] It is understandable that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0251] The memory in the embodiments of the present application can be a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a register, a hard disk, a mobile hard disk, a compact disc read-only memory (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. The storage medium can also be a component of the processor.

[0252] Based on the same technical concept, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program or instruction is stored. When the computer program or instruction is executed by a processor, the computer executes the above-mentioned information processing method.

[0253] Based on the same technical concept, an embodiment of the present invention further provides a computer-readable program product, which, when executed, enables a computer to execute the above information processing method.

[0254] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0255] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0256] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0257] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0258] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An information processing method, characterized in that: The method comprises: Acquire a first question item and a reference answer text, wherein the reference answer text includes a standard text for answering the first question item and a first scoring standard, wherein the first scoring standard includes scoring points and red line points in the standard text, wherein the scoring points are used to indicate a correct answer content, and the red line points are used to indicate an incorrect answer content; receiving a first answer text input by a user, wherein the first answer text is a text obtained by the user answering the first question; The first answer text is matched with the red line point. If the match is successful, a first score is output. If the match fails, a second score is output based on the first answer text and the score point, and the second score is higher than the first score.

2. The method according to claim 1, characterized in that Outputting a second score according to the first answer text and the score point includes: The first answer text and the score point are input into a large language model, a second score output by the large language model is obtained, and the second score is output to the user.

3. The method according to claim 2, characterized in that The method further comprises: Acquire multiple second answer texts input by the user, wherein the multiple second answer texts all fail to match the red line points in the first scoring standard; Obtaining third scores of the plurality of second answer texts input by the professional; Inputting the plurality of second answer texts and the scoring points in the first scoring standard into the large language model, and obtaining a fourth score output by the large language model, and if the difference between the fourth score and the third score is greater than a set value, obtaining an adjusted first scoring standard input by a professional, and repeating this step based on the adjusted first scoring standard until the difference between the fourth score and the third score is less than or equal to the set value; The scores of other answer texts for the first question are determined using the adjusted first scoring standard.

4. The method according to claim 3, characterized in that The step of obtaining a plurality of second answer texts input by a plurality of users comprises: Obtain N third answer texts input by multiple users, where N is an integer greater than or equal to 2; The plurality of second answer texts are sampled from the N third answer texts according to a set sampling rule, and the number of the plurality of second answer texts is the set sampling number.

5. The method according to claim 4, characterized in that The step of sampling the plurality of second answer texts from the N third answer texts according to the sampling rule comprises: Selecting a first number of second answer texts that meet word count requirements, keyword requirements, and screening rule requirements from the N third answer texts; Selecting a second number of second answer texts from the N third answer texts, where the similarity between the third answer text and the reference answer text is greater than a first threshold; Selecting a third number of second answer texts from the N third answer texts, which satisfy that the cosine similarity between the third answer text and the reference answer text is greater than a second threshold; The sum of the first number, the second number and the third number is equal to the set sampling number.

6. The method according to any one of claims 2 to 5, characterized in that The first scoring standard also includes a scoring rule, and the scoring rule includes one or more of the following: a text rule, a regular matching rule, and a prompt word rule; The matching of the first answer text with the red line point includes: Matching the first answer text with the red line point by using the text rule or the regular matching rule; The step of inputting the first answer text and the score points into a large language model comprises: The first answer text, the scoring points and the prompt word rules are input into a large language model.

7. The method according to any one of claims 1 to 5, characterized in that The receiving of the first answer text input by the user comprises: Acquire voice information input by the user, and convert the voice information into first text information; Segmenting the first text information, and replacing the segmented words in the first text information with a first hot word library to obtain second text information, wherein the first hot word library is a hot word library of natural language; The second text information is segmented, and the segmented words in the second text information are replaced using a second hot word library to obtain the first answer text, where the second hot word library is a characteristic hot word library of the field to which the first question belongs.

8. An information processing device, characterized in that: The information processing device comprises: an acquisition module, configured to acquire a first question item and a reference answer text, wherein the reference answer text includes a standard text for answering the first question item and a first scoring standard, wherein the first scoring standard includes scoring points and red line points in the standard text, wherein the scoring points are used to indicate a correct answer content, and the red line points are used to indicate an incorrect answer content; A receiving module, configured to receive a first answer text input by a user, wherein the first answer text is a text obtained by the user answering the first question; A matching module, used for matching the first answer text with the red line point; The output module is used to output a first score if the match is successful, and to output a second score according to the first answer text and the score points if the match fails, wherein the second score is higher than the first score.

9. An information processing device, characterized in that: include: A processor, wherein the processor is coupled to a memory, the memory is used to store a computer program or an instruction, and the processor is used to execute the computer program or the instruction to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program executable by a computer device, and when the program is run on the computer device, the computer device executes the steps of any method described in claims 1 to 7.

11. A computer program product, characterized in that When the method is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images