Intelligent examination scoring method, device, equipment, medium and program product

Abstract

The application provides an intelligent examination scoring method and device, equipment, medium and program product, and relates to the technical field of computers. The method comprises the following steps: obtaining a classification result of supporting materials through a title classification model; obtaining a text abstract of unstructured supporting materials through a text abstract extraction model; obtaining a matching probability of unstructured examination indicators and the text abstract through a text semantic matching double-tower model, and obtaining a score of the unstructured supporting materials according to the matching probability; obtaining key data information of structured supporting materials through a named entity recognition model; obtaining a score of the structured supporting materials according to a size relationship between a first value and a second value corresponding to a first matter in structured examination indicators and the key data information; and multiplying the score of the unstructured supporting materials and the score of the structured supporting materials by a set weight value respectively, adding the two results, and obtaining an examination score.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an intelligent assessment and scoring method, apparatus, device, medium, and program product. Background Technology

[0002] With the continuous development of network technology, the application of deep learning technology is receiving increasing attention. In work applications, it can leverage its intelligent features to assist staff in completing some assessment and evaluation tasks.

[0003] In traditional intelligent assessment and scoring processes, assessors need to read multiple supporting documents and then calculate and score based on different assessment indicators and the corresponding supporting documents.

[0004] However, due to the large number of supporting materials uploaded, examiners need to spend a lot of time and energy reviewing them, which can lead to errors in the subsequent assessment scores and reduce the efficiency of the assessment work. Summary of the Invention

[0005] This application provides an intelligent assessment and scoring method, apparatus, device, medium, and program product, which can improve the efficiency of product design. The technical solution is as follows:

[0006] In one aspect, an intelligent assessment and scoring method is provided, the method comprising:

[0007] The management terminal displays the assessment indicator input interface, which includes a first assessment indicator setting item and a second assessment indicator setting item.

[0008] Receive the structured assessment indicators set through the first assessment indicator setting item and the unstructured assessment indicators set through the second assessment indicator setting item sent by the management terminal;

[0009] The user terminal displays a supporting materials upload interface, which includes supporting materials input fields.

[0010] Receive multiple supporting documents sent by the user terminal and input through the supporting document input field;

[0011] The titles of the multiple supporting materials are respectively input into a BERT-based title classification model to obtain the classification results of the multiple supporting materials output by the title classification model. Based on the classification results, the multiple supporting materials are divided into structured supporting materials and unstructured supporting materials.

[0012] The unstructured supporting material is input into a BERTSUM-based text summarization extraction model to obtain the text summary of the unstructured supporting material output by the text summarization extraction model.

[0013] The unstructured assessment indicators and the text summary are input into a BERT-based dual-tower text semantic matching model to obtain the matching probability between the unstructured assessment indicators and the text summary output by the dual-tower text semantic matching model. The score of the unstructured supporting material is obtained based on the matching probability.

[0014] The structured supporting evidence is input into a BERT-based named entity recognition model to obtain the key data information of the structured supporting evidence output by the named entity recognition model.

[0015] Obtain the first value corresponding to the first item in the structured assessment indicators;

[0016] Obtain the second value corresponding to the first item in the key data information;

[0017] The score of the structured supporting material is obtained based on the relationship between the first value and the second value;

[0018] The scores of the unstructured testimony and the structured testimony are multiplied by a set weight and then added together to obtain the assessment scores of multiple pieces of supporting evidence.

[0019] On the other hand, an intelligent assessment and scoring device is provided, the device comprising:

[0020] The assessment indicator input interface display module is used to display the assessment indicator input interface through the management terminal. The assessment indicator input interface includes a first assessment indicator setting item and a second assessment indicator setting item.

[0021] The performance indicator acquisition module is used to receive structured performance indicators set through the first performance indicator setting item and unstructured performance indicators set through the second performance indicator setting item sent by the management terminal.

[0022] The supporting materials upload interface display module is used to display the supporting materials upload interface through the user terminal. The supporting materials upload interface includes supporting materials input items.

[0023] The supporting materials acquisition module is used to receive multiple supporting materials sent by the user terminal and input through the supporting materials input field;

[0024] The supporting material segmentation module is used to input the titles of multiple supporting materials into a BERT-based title classification model, obtain the classification results of each of the multiple supporting materials output by the title classification model, and classify the multiple supporting materials into structured supporting materials and unstructured supporting materials according to the classification results.

[0025] The text summarization module is used to input the unstructured supporting material into a BERTSUM-based text summarization extraction model and obtain the text summary of the unstructured supporting material output by the text summarization extraction model.

[0026] The first score acquisition module is used to input the unstructured assessment indicators and the text summary into a BERT-based text semantic matching dual-tower model, obtain the matching probability between the unstructured assessment indicators and the text summary output by the text semantic matching dual-tower model, and obtain the score of the unstructured supporting material based on the matching probability.

[0027] The key data information acquisition module is used to input the structured supporting materials into a BERT-based named entity recognition model and acquire the key data information of the structured supporting materials output by the named entity recognition model.

[0028] The first value acquisition module is used to acquire the first value corresponding to the first item in the structured assessment indicators;

[0029] The second value acquisition module is used to acquire the second value corresponding to the first item in the key data information;

[0030] The second score acquisition module is used to acquire the score of the structured supporting material based on the relationship between the first value and the second value.

[0031] The final score acquisition module is used to multiply the scores of the unstructured testimony materials and the scores of the structured testimony materials by a set weight and add them together to obtain the assessment scores of multiple supporting evidence materials.

[0032] In another aspect, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the above-mentioned intelligent assessment and scoring method.

[0033] In another aspect, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the above-mentioned intelligent assessment and scoring method.

[0034] In another aspect, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the intelligent assessment and scoring methods provided in the various optional implementations described above.

[0035] The technical solutions provided in this application embodiment may have the following beneficial effects:

[0036] The computer device can display the assessment indicator input interface of the intelligent assessment system. After receiving setting operations for the first and second assessment indicator setting items displayed on the assessment indicator input interface, it acquires structured and unstructured assessment indicators. The computer device can also display the supporting material upload interface of the intelligent assessment system. After receiving operations for the supporting material input items displayed on the supporting material upload interface, it acquires structured and unstructured supporting materials. Subsequently, through the backend of the intelligent assessment system, the computer device, based on a title classification model, a text summary extraction model, a text semantic matching dual-tower model, and a named entity recognition model, respectively acquires the classification results of the supporting materials, the matching probability between the unstructured assessment indicators and the text summary, and the key data information of the structured supporting materials. Then, based on the matching... The system obtains scores for unstructured supporting materials based on probability. It then calculates scores for structured supporting materials based on the relationship between the first value corresponding to the first item in the structured assessment indicators and the second value corresponding to the first item in the key data information. Finally, it calculates the assessment score for the supporting materials based on both the unstructured and structured scores. In this approach, when assessing supporting materials, staff only need to set the parameters for both structured and unstructured assessment indicators in the assessment indicator input interface of the management terminal. The server can then automatically calculate the score based on different models combined with the user-uploaded supporting materials. This process is easy to understand and operate, requiring minimal effort from designers to master the assessment scoring rules, thus significantly improving assessment efficiency. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application;

[0038] Figure 2 This is a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application;

[0039] Figure 3 This is a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application;

[0040] Figure 4This is a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application;

[0041] Figure 5 This is a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application;

[0042] Figure 6 This is a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application;

[0043] Figure 7 This is a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application;

[0044] Figure 8 This is a flowchart of an intelligent assessment and scoring method based on deep learning technology according to one embodiment of this application;

[0045] Figure 9 This is a structural diagram of a BERT-based title classification model according to one embodiment of this application;

[0046] Figure 10 This is a structural diagram of a BERTSUM-based text summarization extraction model according to an embodiment of this application;

[0047] Figure 11 This is a structural diagram of a BERT-based dual-tower model for text semantic matching, according to one embodiment of this application.

[0048] Figure 12 This is a structural diagram of a BERT-based named entity recognition model according to one embodiment of this application;

[0049] Figure 13 This is a block diagram of an intelligent assessment and scoring device provided in one embodiment of this application;

[0050] Figure 14 This is a structural block diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0052] 1) Supporting materials: Materials used to support or prove the authenticity and accuracy of other supporting materials. These can be data, information, documents, records, or other relevant information. Supporting materials can be divided into structured and unstructured supporting materials. Structured supporting materials represent those that can be quantified, while unstructured supporting materials represent those that cannot be quantified.

[0053] 2) Performance indicators: These are standards used to measure an individual's performance and achievements in a specific task or goal. Performance indicators can be divided into structured and unstructured indicators. Structured indicators represent those that can be quantified, while unstructured indicators represent those that cannot be quantified.

[0054] 3) BERT (Bidirectional Encoder Representations from Transformers) model: A natural language processing model whose main purpose is to provide deep semantic understanding and representation learning for text data.

[0055] 4) BERTSUM (BERT Extractive Summarizer) model: A text summarization tool based on the BERT model, used to automatically generate summaries or conclusions of text.

[0056] Figure 1 A schematic diagram of an implementation environment provided by an exemplary embodiment of this application is shown. This implementation environment may include: a first terminal 110, a second terminal 120, and a server 130.

[0057] exist Figure 1 In computer system 100, the first terminal 110 is the terminal used by the administrator to set performance indicators, and the second terminal 120 is the terminal used by the user to upload supporting materials.

[0058] The first terminal 120 and the second terminal 120 may have the same or different device types, including at least one of smartphones, tablets, laptops, and desktop computers.

[0059] The first terminal 110 and the second terminal 120 are connected to the server 130 via a wireless network or a wired network, respectively.

[0060] Please refer to Figure 2 The document illustrates a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application. For ease of explanation, the description focuses solely on the server as the entity executing each step; this server may be... Figure 1 The server 130 shown; the method may include the following steps.

[0061] Step 200: Display the assessment indicator input interface through the management terminal. The assessment indicator input interface includes the first assessment indicator setting item and the second assessment indicator setting item.

[0062] Step 210: Receive the structured assessment indicators set through the first assessment indicator setting item and the unstructured assessment indicators set through the second assessment indicator setting item sent by the management terminal.

[0063] Step 220: Display the supporting materials upload interface through the user terminal. The supporting materials upload interface includes supporting materials input fields.

[0064] Step 230: Receive multiple supporting documents sent by the user terminal, entered through the supporting documents input field.

[0065] Step 240: Input the titles of multiple supporting materials into the BERT-based title classification model to obtain the classification results of each supporting material output by the title classification model. Based on the classification results, divide the multiple supporting materials into structured supporting materials and unstructured supporting materials.

[0066] Step 250: Input the unstructured supporting material into the BERTSUM-based text summarization extraction model to obtain the text summary of the unstructured supporting material output by the text summarization extraction model.

[0067] Step 260: Input the unstructured assessment indicators and text summary into the BERT-based dual-tower text semantic matching model, obtain the matching probability between the unstructured assessment indicators and the text summary output by the dual-tower text semantic matching model, and obtain the score of the unstructured supporting materials based on the matching probability.

[0068] Step 270: Input the structured supporting evidence into the BERT-based named entity recognition model and obtain the key data information of the structured supporting evidence output by the named entity recognition model.

[0069] Step 280: Obtain the first value corresponding to the first item in the structured assessment indicators.

[0070] Step 290: Obtain the second value corresponding to the first item in the key data information.

[0071] Step 300: Based on the relationship between the first and second values, obtain the score of the structured supporting material.

[0072] Step 310: Multiply the scores of unstructured testimony materials and structured testimony materials by the set weights and add them together to obtain the assessment scores of multiple supporting evidence materials.

[0073] The solution shown in this application embodiment allows the computer device to display the assessment indicator input interface of the intelligent assessment system. After receiving setting operations for the first and second assessment indicator setting items displayed in the assessment indicator input interface, the computer device acquires structured and unstructured assessment indicators. The computer device can also display the supporting material upload interface of the intelligent assessment system. After receiving operations for the supporting material input items displayed in the supporting material upload interface, the computer device acquires structured and unstructured supporting materials. Subsequently, through the background of the intelligent assessment system, the computer device acquires the classification results of the supporting materials, the matching probability between the unstructured assessment indicators and the text summary, and the key data information of the structured supporting materials based on the title classification model, text summary extraction model, text semantic matching dual-tower model, and named entity recognition model, respectively. Then, based on the matching probability, the computer device acquires the score of the unstructured supporting materials. Based on the relationship between the first value corresponding to the first item in the structured assessment indicators and the second value corresponding to the first item in the key data information, the computer device acquires the score of the structured supporting materials. Finally, the computer device obtains the assessment score of the supporting materials based on the scores of the unstructured and structured supporting materials.

[0074] In summary, in this embodiment of the application, when it is necessary to assess and score supporting materials, staff only need to set the parameters of structured and unstructured assessment indicators in the assessment indicator input interface of the management terminal. Subsequently, the server can automatically calculate the score based on different machine learning models combined with the supporting materials uploaded by the user. The above operation steps are easy to understand and operate, and do not require designers to invest a lot of effort to master and become familiar with the assessment and scoring rules, which can greatly improve the efficiency of assessment and scoring.

[0075] based on Figure 2 Please refer to the proposed solution. Figure 3 This illustrates a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application. Figure 3 As shown, Figure 2 In the illustrated embodiment, steps 231, 232, 233, and 234 may be included before step 240.

[0076] Step 231: Obtain multiple supporting material title samples, and the category label added to each supporting material title sample; the category label is used to indicate whether the supporting material corresponding to the supporting material title sample is structured supporting material or unstructured supporting material.

[0077] The above category labels can be divided into meeting materials, work summaries, systems and mechanisms, etc., to ensure that the number of samples in each category in the training set is roughly the same.

[0078] Step 232: Input the obtained supporting material title sample into the BERT-based title classification model to obtain a predicted probability distribution. The predicted probability distribution includes a first probability and a second probability. The first probability represents the probability that the supporting material sample corresponding to the supporting material title sample is structured supporting material, and the second probability represents the probability that the supporting material sample corresponding to the supporting material title sample is unstructured supporting material.

[0079] The aforementioned BERT-based title classification model consists of three main modules: a data input module, a feature extraction module, and a classification module. The feature extraction module uses the BERT model to extract text features.

[0080] The server first compares the sample titles of the supporting materials with their corresponding tags (x). title y cls The input is fed into the input module, which encodes the title samples of the supporting materials into a low-dimensional vector. The encoding consists of two main parts: category and word vector encoding. Word vector encoding involves segmenting the text content to obtain tokens, then using the fixed vocabulary of the WordPiece model to map the tokens to corresponding token IDs. The encoded vector is then input into the feature extraction module for feature extraction to obtain the feature vector Y of the sentence. pred The feature extraction module requires preloading the weight parameters of the pre-trained BERT model.

[0081] Step 233: Calculate the first loss using the cross-entropy loss function based on the predicted probability distribution and the probability of the category labels added to the title samples of the supporting materials.

[0082] The Softmax function is part of a title classification model, specifically the output layer. After processing the titles, the title classification model can output a probability distribution using the Softmax function. The formula for the Softmax function is shown in Formula 1.

[0083]

[0084] Where C is the set of categories for the titles of the supporting materials for assessment, z i This indicates the current material model output score, z. j P represents the output score of the current material in different categories. i This represents the probability value of the i-th category in the probability distribution generated after the current material is processed by the Softmax function.

[0085] The cross-entropy loss function is used to measure the difference between the model output and the actual label. The formula for the cross-entropy loss function is shown in Formula 2 below:

[0086] L=-∑ i∈Cy i log(P i (2)

[0087] Where y i Let L represent the label value of the i-th category in the actual labels, and L represent the loss value, which is used to measure the difference between the model's output probability value and the actual label probability value. The smaller this difference is, the smaller the loss value is, indicating that the model's predicted probability value is closer to the actual label probability value.

[0088] Step 234: Based on the first loss, update the parameters of the BERT-based title classification model using the gradient descent algorithm.

[0089] Repeat the above steps until the first loss converges to the preset condition.

[0090] In this embodiment of the application, by combining the title samples of the supporting materials for the assessment with the corresponding labels and training the BERT-based title classification model with the cross-entropy function, the noise of label assignment can be reduced, and the model can benefit from training samples with correct labels, thereby improving classification accuracy. On the other hand, the language understanding ability of BERT can be fully utilized to improve the performance of the model.

[0091] based on Figure 2 or Figure 3 Please refer to the proposed solution. Figure 4 This illustrates a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application. Figure 4 As shown, Figure 2 or Figure 3 In the illustrated embodiment, steps 241, 242, 243, and 244 may be included before step 250.

[0092] Step 241: Perform preprocessing on the unstructured evidence sample, adding CLS and SEP tags to each sentence in the unstructured evidence sample.

[0093] The above preprocessing operations can be implemented through code. CLS corresponds to the word vector of the first word in the input text, and SEP corresponds to the word vector of the last word in the input text. After adding CLS and SEP tags to each sentence in the unstructured supporting material sample, they can be input into the model and trained.

[0094] Step 242: Input the preprocessed unstructured supporting material samples into the BERTSUM-based text summarization extraction model to obtain a predicted text summary.

[0095] The BERTUM-based text summarization extraction model described above consists of three main modules: a data input module, a feature extraction module, and a summarization extraction module. The feature extraction module uses the BERT model to extract text features, while the summarization extraction module employs a transform+Sigmoid approach.

[0096] First, the unstructured evidence material samples are input into the input module of the constructed BERTSUM text summarization extraction model to encode the evidence materials into a low-dimensional space vector. The word vector encoding is to segment the text content to obtain tokens. Then, the fixed vocabulary of the WordPiece model is used to map the tokens to the corresponding token IDs. The encoded vector is then input into the feature extraction module to extract the feature vector of the sentence. The feature extraction module needs to preload the weight parameters of the pre-trained BERT model. Finally, the feature vector is input into the transform+Sigmoid module, and the formulas are shown in formulas (3), (4), and (5).

[0097]

[0098]

[0099]

[0100] in, h represents the value of the l-th hidden state obtained after passing through the (L-1)-th hidden state. l-1 h represents the hidden state value at layer L-1. l This represents the hidden state value of layer L. b represents the weight value. o This represents the bias value, where σ represents the sigmoid function. The classification result is represented by MMHAtt, which indicates the multi-head attention mechanism, and LN represents the regularization operation.

[0101] Step 243: Calculate the second loss using a loss function based on the predicted text summary and the text summary corresponding to the unstructured supporting material sample.

[0102] Specifically, a second loss can be calculated based on the differences between the predicted text summary and the text summaries corresponding to the unstructured supporting material samples. For example, semantic features of the predicted text summary and the text summaries corresponding to the unstructured supporting material samples can be extracted, and then the semantic features of the predicted text summary and the text summaries corresponding to the unstructured supporting material samples can be input into the loss function to calculate the second loss based on the differences between the semantic features of the predicted text summary and the text summaries corresponding to the unstructured supporting material samples.

[0103] Step 244: Based on the second loss, update the parameters of the BERTSUM-based text summarization extraction model using an optimization algorithm.

[0104] Repeat the above steps until the second loss converges to the preset condition.

[0105] In this embodiment of the application, the text summarization extraction model based on BERTSUM is trained by combining unstructured evidence material samples with a loss function. Unstructured evidence material samples usually contain different text forms and contents. The model can learn different text structures and word usage, improve the model's generalization ability, and thus improve the model's efficiency in assessment and scoring tasks.

[0106] based on Figures 2 to 4 Please refer to any of the proposed solutions. Figure 5 This illustrates a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application. Figure 5 As shown, Figures 2 to 4 Before step 260 in any of the illustrated embodiments, steps 251, 252, 253, and 254 may also be included.

[0107] Step 251: Obtain unstructured assessment indicator samples, unstructured supporting material text summary samples corresponding to each unstructured assessment indicator sample, and matching tags. The matching tags are used to indicate the degree of matching between the unstructured assessment indicator samples and their corresponding unstructured supporting material text summary samples.

[0108] The above-mentioned assessment indicator sample can be a descriptive text containing the assessment points and material submission requirements. It also includes keywords added based on the characteristics of specific supporting materials, such as important file names mentioned in the materials. The above-mentioned text summary sample covers the main content of the supporting materials.

[0109] The above-mentioned assessment indicator samples and their corresponding text summary samples are combined into matching text pairs, and a matching label is added to each matching text pair. The matching label can be either a match or a non-match. A matching label indicates that the sample pair is a positive sample pair, and a non-match label indicates that the sample pair is a negative sample pair. A training set consisting of a 1:1 ratio of positive samples to negative samples is constructed.

[0110] Step 252: Input the unstructured assessment indicator samples and their corresponding unstructured supporting material text summary samples into the BERT-based text semantic matching dual-tower model to obtain a predicted matching degree.

[0111] The aforementioned BERT-based dual-tower text semantic matching model consists of three main modules: a data input module, a feature extraction module, and a feature matching module. The feature extraction module uses the BERT model to extract text features.

[0112] Pair the above assessment indicator samples with their corresponding text summary samples (x cri ,x sup The text content information is input into the input module and encoded into a low-dimensional space vector. The encoding mainly consists of two parts: word vector encoding and position vector encoding. Word vector encoding involves segmenting the text content into words to obtain tokens, and then using the fixed vocabulary of the WordPiece model to map the tokens to the corresponding token IDs. Position vector encoding adopts the relative position encoding method, as shown in formula (6).

[0113]

[0114] in For word vectors, R i-j For relative positions and ij>0, u T v T W is a trainable weight matrix. Finally, the encoded word vectors and position vectors are summed to obtain the final vector.

[0115] The encoded vector is then input into the feature extraction module to extract features and obtain the feature vector (y) of the statement. cri ,y sup ), where cri represents the feature vector of the assessment indicator and sup represents the feature vector of the testimony material.

[0116] The feature extraction module needs to preload the weight parameters of the pre-trained BERT model, and finally calculate the cosine matching degree of the two feature vectors, as shown in formula (7):

[0117]

[0118] Step 253: Calculate the third loss using a loss function based on the probability of predicted matching degree and the probability of matching label.

[0119] The loss of the model is calculated using the cross-entropy loss function, where the formula for the cross-entropy function is shown in Equation (8):

[0120]

[0121] Where γ is a smoothing coefficient of the cross-entropy function, and y is a hyperparameter. sup It is a set of supporting materials corresponding to an assessment indicator, including all positive samples of supporting materials and four random negative samples. The cross-entropy loss function formula is shown in formula (9):

[0122]

[0123] in This indicates a positive sample of supporting evidence.

[0124] Step 254: Based on the third loss, update the parameters of the BERT-based dual-tower text semantic matching model using an optimization algorithm.

[0125] Repeat the above steps until the third loss converges to the preset condition.

[0126] In this embodiment, a sample pair consisting of unstructured assessment indicator samples and their corresponding unstructured supporting material text summary samples is used to train a BERT-based dual-tower text semantic matching model using a cross-entropy loss function. Because the BERT model has context-aware capabilities and can understand contextual information in the text, the dual-tower model can better understand the context of the assessment indicators and supporting material texts by training with BERT, thereby capturing semantic matching more accurately. On the other hand, the cross-entropy loss function usually has good numerical stability, which helps to avoid numerical problems such as gradient vanishing or gradient exploding, making training more reliable and thus ensuring the accuracy of the model.

[0127] based on Figures 2 to 5 Please refer to any of the proposed solutions. Figure 6 This illustrates a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application. Figure 6 As shown, Figures 2 to 6 In any of the embodiments shown, steps 261, 262, 263, and 264 may be included before step 270.

[0128] Step 261: Obtain structured supporting material samples and annotation information for each structured supporting material sample. The annotation information is used to indicate the key fields corresponding to the assessment indicators contained in the structured supporting material samples.

[0129] The above annotation information is used to annotate named entities according to specific needs, such as time, location, number of meetings, etc.

[0130] Step 262: Input the structured supporting material sample into the BERT-based named entity recognition model to obtain a predicted annotation information.

[0131] The aforementioned BERT-based named entity recognition model mainly consists of three modules: a data input module, a feature extraction module, and a named entity recognition module. The feature extraction module uses the BERT model to extract text features, while the named entity recognition module adopts a linear chain conditional random field model.

[0132] First, the sample pairs (x) consisting of the above-mentioned supporting materials and corresponding labels are selected. supThe tokens y and y are input to the input module, which encodes the text content into a low-dimensional vector. Then, using the fixed vocabulary of the WordPiece model, the tokens are mapped to their corresponding token IDs. Finally, the encoded vectors are input to the feature extraction module to extract the feature vector y of the sentence. pred The feature extraction module requires preloading the weight parameters of the pre-trained BERT model, and finally extracts the feature vector y. pred The input named entity module, that is, the feature vector is input into the conditional random field model, the model predicts the label for each token, obtains the label sequence, maps the label sequence back to the original text, and obtains a predicted label information.

[0133] Step 263: Based on the predicted annotation information and the annotation information corresponding to the structured supporting material samples, calculate the fourth loss using the log-likelihood loss function.

[0134] In this embodiment of the application, the fourth loss can be calculated using the log-likelihood loss function, that is, the loss of the model can be calculated using the linear chain conditional random field, as shown in formulas (10) and (11):

[0135]

[0136]

[0137] in and These represent the i-th label y in the label sequence y corresponding to the supporting materials. i The EmissionScore and TransitionScore.

[0138] Step 264: Based on the fourth loss, update the parameters of the BERT-based named entity recognition model using an optimization algorithm.

[0139] Repeat the above steps until the fourth loss converges to the preset condition.

[0140] In this embodiment, the BERT-based named entity recognition model is trained by combining the sample pairs consisting of supporting material samples and corresponding labels with the log-likelihood loss function. This can improve model performance and generalization ability, reduce label noise, improve contextual understanding, reduce the risk of overfitting, and ensure the accuracy of subsequent assessment and scoring tasks.

[0141] based on Figures 2 to 6 Please refer to any of the proposed solutions. Figure 7 This illustrates a flowchart of an intelligent assessment and scoring method provided in one embodiment of this application. Figure 7 As shown, Figures 2 to 6In any of the embodiments shown, step 300 can be implemented as step 300a.

[0142] Step 300a: In response to the first value being greater than the second value, subtract a preset score from the score of the structured supporting material.

[0143] The first value mentioned above is the value corresponding to the first item in the structured assessment indicators, and the second value is the value corresponding to the first item in the key data information. The two values ​​are compared. If the first value is greater than the second value, a preset score is deducted from the score of the structured supporting materials.

[0144] For example, the structured performance evaluation criteria required five meetings this year, but analysis of the key data in the materials showed that only four meetings were held. According to the evaluation rules, a certain number of points were deducted.

[0145] In this embodiment of the application, a preset score is deducted from the score of the structured supporting materials based on numerical comparison, which makes the calculation of the assessment score more convenient and improves the efficiency of the assessment scoring.

[0146] In combination with the above Figures 2 to 7 The illustrated scheme, taking an intelligent assessment and scoring method based on deep learning technology as an example, provides a detailed description of specific embodiments. It should be understood that the specific implementation methods described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of these embodiments.

[0147] Figure 8 This is a flowchart of an intelligent assessment and scoring method based on deep learning technology according to an embodiment of this application. The method includes:

[0148] Step 1: The assessment indicators are manually divided into structured indicators and unstructured indicators. Supporting materials are input into the title classification model and automatically divided into structured supporting materials and unstructured supporting materials.

[0149] Step 2: For unstructured supporting materials, input them into the text summarization model to obtain a text summary of the supporting materials. The text summary can also serve as a reference for investigators.

[0150] Step 3: Input the unstructured indicators and the text summaries of the supporting materials from Step 2 into the dual-tower model to obtain the matching probability between the unstructured indicators and the text summaries of the supporting materials, and use this probability as the score of the unstructured supporting materials.

[0151] Step 4: For structured supporting materials, input them into the information extraction model to extract key data information from the supporting materials;

[0152] Step 5: Compare the key data information of the supporting materials in Step 4 with the structured indicators and perform statistical analysis to calculate the score of the structured supporting materials;

[0153] Step Six: Multiply the scores of the unstructured supporting materials in Step Three and the scores of the structured supporting materials in Step Five by the set weights and add them together to obtain the final score of the assessment indicator.

[0154] Figure 9 This is a structural diagram of a BERT-based title classification model involved in one embodiment of the application. The supporting material title classification model is constructed in the following manner:

[0155] Step 1: Collect the titles of supporting materials for the assessment and add category labels to each title. The labels for supporting materials include meeting materials, work summaries, and institutional mechanisms. Divide the constructed dataset into training, validation, and test sets. To reduce the influence of the training set on the model, try to ensure that the number of samples in each category in the training set is roughly the same.

[0156] Step 2: Construct a BERT-based title classification model, which mainly consists of three modules: a data input module, a feature extraction module, and a classification module. The feature extraction module uses the BERT model to extract text features.

[0157] Step 3: Train the BERT-based title classification model built in Step 2 on the training set from Step 1. First, input the title of the supporting materials and its corresponding labels into the input module. The main function of the input module is to encode the title of the supporting materials into a low-dimensional vector space. The encoding mainly consists of two parts: category and word vector encoding. Word vector encoding involves segmenting the text content to obtain tokens, and then using the fixed vocabulary of the WordPiece model to map the tokens to their corresponding token IDs. Next, the encoded vector is input into the feature extraction module to extract the feature vector of the sentence. The feature extraction module needs to pre-load the weight parameters of the pre-trained BERT model. Finally, the cross-entropy loss function is used to calculate the model's loss.

[0158] Step 4: Input the titles and labels of the supporting materials in the validation set into the model trained in Step 3 to obtain the classification results of the supporting materials.

[0159] Figure 10 This is a structural diagram of a BERTSUM-based text summarization extraction model according to an embodiment of this application. The supporting material text summarization extraction model is constructed in the following manner:

[0160] Step 1: Preprocess the unstructured supporting materials by adding CLS and SEP tags before and after each sentence in the materials.

[0161] Step 2: Construct the BERTSUM text summarization extraction model, which mainly consists of three modules: a data input module, a feature extraction module, and a summarization extraction module. The feature extraction module uses the BERT model to extract text features, and the summarization extraction module uses the transform+Sigmoid module.

[0162] Step 3: Input the preprocessed supporting materials from Step 1 into the input module of the BERTSUM text summarization extraction model built in Step 2. The main function of the input module is to encode the supporting materials into low-dimensional vectors. Word vector encoding involves segmenting the text content to obtain tokens, and then using the fixed vocabulary of the WordPiece model to map the tokens to the corresponding token IDs. The encoded vectors are then input into the feature extraction module to extract the feature vectors of the sentences. The feature extraction module needs to preload the weight parameters of the pre-trained BERT model. Finally, the feature vectors are input into the transform+Sigmoid module.

[0163] Step 4: Sort the results from Step 3 and select the top K sentences as the text summary of the supporting material for this assessment.

[0164] Figure 11 This is a structural diagram of a BERT-based dual-tower model for text semantic matching, according to one embodiment of this application. The evaluation metrics and supporting materials in the dual-tower model for text semantic matching are constructed in the following way:

[0165] Step 1: Collect unstructured assessment indicators and their corresponding unstructured supporting material summaries. Assessment indicators are mostly descriptive texts, mainly containing assessment points and material submission requirements. Keywords are added based on the characteristics of some special supporting materials, such as important filenames mentioned in the materials. The supporting material summaries are condensed and refined, covering the main content of the supporting materials. The assessment indicators and their corresponding supporting material summaries are paired into matching text pairs, and each pair is labeled as either matched or unmatched. This means constructing not only positive sample pairs but also negative sample pairs. To reduce the influence of the training set on the model, the ratio of positive to negative samples in the training set should be maintained at approximately 1:1. Finally, the dataset is divided into training, validation, and test sets.

[0166] Step 2: Construct a BERT-based dual-tower text semantic matching model, which mainly consists of three modules: a data input module, a feature extraction module, and a feature matching module. The feature extraction module uses the BERT model to extract text features.

[0167] Step 3: Train the BERT-based dual-tower text semantic matching model built in Step 2 on the training set from Step 1. First, input the evaluation metrics and supporting material summaries into the input module. The main function of the input module is to encode the text content information from Step 1 into a low-dimensional vector space. The encoding mainly consists of two parts: word vector encoding and position vector encoding. Word vector encoding involves segmenting the text content to obtain tokens, and then using the fixed vocabulary of the WordPiece model to map the tokens to their corresponding token IDs. Position vector encoding uses a relative position encoding method, and finally, the encoded word vectors and position vectors are summed to obtain the final vector. Then, the encoded vectors are input into the feature extraction module to extract the feature vectors of the sentences. The feature extraction module needs to preload the weight parameters of the pre-trained BERT model and calculate the cosine matching degree of the two feature vectors. Finally, the cross-entropy loss function is used to calculate the model loss.

[0168] Step 4: Input the validation set assessment indicators and all supporting material summaries into the model trained in Step 3 in sequence to obtain the supporting material summary matching probability and use it as the score of the unstructured assessment indicator.

[0169] Figure 12 This is a structural diagram of a BERT-based named entity recognition model according to an embodiment of this application. The key indicator field value recognition model in the supporting materials is constructed in the following way:

[0170] Step 1: Collect structured supporting materials and label them according to the key fields in the structured assessment indicators. Divide the constructed dataset into training set, validation set and test set.

[0171] Step 2: Construct a BERT-based named entity recognition model, which mainly consists of three modules: a data input module, a feature extraction module, and a named entity recognition module. The feature extraction module uses the BERT model to extract text features, and the named entity recognition module employs a linear chain conditional random field (CLR) module.

[0172] Step 3: Train the BERT-based named entity recognition model from Step 2 on the training set from Step 1. First, input the supporting materials and corresponding labels into the input module. The main function of the input module is to encode the text content information from Step 1 into a low-dimensional vector space. The encoding mainly consists of two parts: category encoding and word vector encoding. Word vector encoding involves segmenting the text content to obtain tokens, and then using the fixed vocabulary of the WordPiece model to map the tokens to the corresponding token IDs. Then, the encoded vector is input into the feature extraction module to extract the feature vector of the sentence. The feature extraction module needs to preload the weight parameters of the pre-trained BERT model. Finally, a linear chain conditional random field is used to calculate the model's loss.

[0173] Step 4: Perform statistical analysis on the key field values ​​of the obtained supporting materials and the assessment indicators to obtain the scores of the structured indicators.

[0174] In this application embodiment, an intelligent assessment and scoring method based on deep learning technology is provided, which realizes intelligent analysis of review materials through artificial intelligence methods, thereby reducing the time and effort of manual review of supporting materials. Compared with traditional manual review and scoring of materials, it greatly reduces the review time and improves the work efficiency of office staff.

[0175] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0176] Please refer to Figure 13 This diagram illustrates a block diagram of an intelligent assessment and scoring device according to an embodiment of this application. The device may include the following modules:

[0177] The assessment indicator input interface display module 1300 is used to display the assessment indicator input interface through the management terminal. The assessment indicator input interface includes a first assessment indicator setting item and a second assessment indicator setting item.

[0178] The performance indicator acquisition module 1301 is used to receive structured performance indicators set through the first performance indicator setting item and unstructured performance indicators set through the second performance indicator setting item sent by the management terminal.

[0179] The supporting materials upload interface display module 1302 is used to display the supporting materials upload interface through the user terminal. The supporting materials upload interface includes supporting materials input items.

[0180] The supporting materials acquisition module 1303 is used to receive multiple supporting materials sent by the user terminal and entered through the supporting materials input field;

[0181] The supporting material segmentation module 1304 is used to input the titles of multiple supporting materials into the BERT-based title classification model, obtain the classification results of each of the multiple supporting materials output by the title classification model, and divide the multiple supporting materials into structured supporting materials and unstructured supporting materials according to the classification results.

[0182] The text summarization module 1305 is used to input unstructured supporting materials into a BERTSUM-based text summarization extraction model and obtain the text summarization of the unstructured supporting materials output by the text summarization extraction model.

[0183] The first score acquisition module 1306 is used to input unstructured assessment indicators and text summaries into the BERT-based text semantic matching dual-tower model, obtain the matching probability between the unstructured assessment indicators and the text summaries output by the text semantic matching dual-tower model, and obtain the score of the unstructured supporting materials based on the matching probability.

[0184] The key data information acquisition module 1307 is used to input structured supporting materials into the BERT-based named entity recognition model and acquire the key data information of the structured supporting materials output by the named entity recognition model.

[0185] The first value acquisition module 1308 is used to acquire the first value corresponding to the first item in the structured assessment indicators;

[0186] The second value acquisition module 1309 is used to acquire the second value corresponding to the first item in the key data information;

[0187] The second score acquisition module 1310 is used to acquire the score of the structured supporting material based on the relationship between the first value and the second value.

[0188] The final score acquisition module 1311 is used to multiply the scores of unstructured testimony materials and structured testimony materials by a set weight and add them together to obtain the assessment scores of multiple supporting materials.

[0189] In some embodiments, the above-described apparatus further includes a title classification model training module, for,

[0190] Obtain multiple sample titles of supporting materials, and add category tags to each sample title; the category tags are used to indicate whether the supporting material corresponding to the sample title is structured or unstructured.

[0191] The obtained supporting material title samples are input into the BERT-based title classification model to obtain a predicted probability distribution. The predicted probability distribution includes a first probability and a second probability. The first probability represents the probability that the supporting material sample corresponding to the supporting material title sample is a structured supporting material, and the second probability represents the probability that the supporting material sample corresponding to the supporting material title sample is an unstructured supporting material.

[0192] The first loss is calculated using the cross-entropy loss function based on the predicted probability distribution and the probability of the category labels added to the title samples of the supporting materials.

[0193] Based on the first loss, the parameters of the BERT-based title classification model are updated using the gradient descent algorithm;

[0194] Repeat the above steps until the first loss converges to the preset condition.

[0195] In some embodiments, the above apparatus further includes a text summarization extraction model training module, used for,

[0196] Preprocess the unstructured evidence samples by adding CLS and SEP tags to each sentence in the unstructured evidence samples.

[0197] Multiple unstructured supporting material samples, after preprocessing, are input into a BERTSUM-based text summarization extraction model to obtain a predicted text summary.

[0198] Based on the predicted text summary and the text summary corresponding to the unstructured supporting material samples, the second loss is calculated using a loss function;

[0199] Based on the second loss, the parameters of the BERTSUM-based text summarization extraction model are updated using an optimization algorithm;

[0200] Repeat the above steps until the second loss converges to the preset condition.

[0201] In some embodiments, the above-described apparatus further includes a text semantic matching dual-tower model training module, used for,

[0202] Obtain unstructured assessment indicator samples, unstructured supporting material text summary samples corresponding to each unstructured assessment indicator sample, and matching tags. The matching tags are used to indicate the degree of matching between the unstructured assessment indicator samples and their corresponding unstructured supporting material text summary samples.

[0203] Input the unstructured assessment indicator samples and their corresponding unstructured supporting material text summary samples into the BERT-based text semantic matching dual-tower model to obtain a predicted matching degree.

[0204] The third loss is calculated based on the probability of predicted matching degree and the probability of matching label using a loss function;

[0205] Based on the third loss, the parameters of the BERT-based dual-tower text semantic matching model are updated using an optimization algorithm;

[0206] Repeat the above steps until the third loss converges to the preset condition.

[0207] In some embodiments, the above apparatus further includes a named entity recognition model training module, used for,

[0208] Obtain structured supporting material samples and annotation information for each structured supporting material sample. The annotation information is used to indicate the key fields corresponding to the assessment indicators contained in the structured supporting material samples.

[0209] The structured supporting material sample is input into the BERT-based named entity recognition model to obtain a predicted annotation information;

[0210] Based on the predicted annotation information and the annotation information corresponding to the structured supporting material samples, the fourth loss is calculated using the log-likelihood loss function;

[0211] Based on the fourth loss, the parameters of the BERT-based named entity recognition model are updated through an optimization algorithm;

[0212] Repeat the above steps until the fourth loss converges to the preset condition.

[0213] In some embodiments, the second score acquisition module 1310 is specifically used for,

[0214] If the first value is greater than the second value, a preset score is deducted.

[0215] Figure 14 This illustration shows a structural block diagram of a computer device 1400 provided in an exemplary embodiment of this application. Examples include personal computers, smartphones, and tablet computers. The computer device 1400 may also be referred to by other names such as user equipment.

[0216] Typically, computer device 1400 includes a processor 1401 and a memory 1402.

[0217] Memory 1402 may include one or more computer-readable storage media, which may be tangible and non-transitory. Memory 1402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in memory 1402 is used to store at least one instruction, which is executed by processor 1401 to implement all or part of the steps performed by computer-aided design software tools in the methods provided in the embodiments of this application.

[0218] In some embodiments, the computer device 1400 may also optionally include: a peripheral device interface 1403 and at least one peripheral device. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1404, a touch display screen 1405, a camera 1406, an audio circuit 1407, and a power supply 1408.

[0219] In some embodiments, the computer device 1400 further includes one or more sensors 1409. The one or more sensors 1409 include, but are not limited to, an accelerometer 1410, a gyroscope 1411, a pressure sensor 1412, an optical sensor 1413, and a proximity sensor 1414.

[0220] Those skilled in the art will understand that the structure shown above does not constitute a limitation on the computer device 1400, and may include more or fewer components than shown, or combine certain components, or employ different component arrangements.

[0221] In one exemplary embodiment, a computer-readable storage medium is also provided for storing at least one computer program, which is loaded and executed by a processor to implement all or part of the steps in the methods shown in the various embodiments above. For example, the computer-readable storage medium may be a read-only memory, a random access memory, a read-only optical disk, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0222] In one exemplary embodiment, a computer program product is also provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform all or part of the steps in the methods shown in the various embodiments above.

[0223] Before and during the collection of user data, this application can display a prompt interface, pop-up window, or output voice prompt information. This prompt interface, pop-up window, or voice prompt information is used to inform the user that their relevant data is being collected. This ensures that the application only begins to execute the relevant steps of collecting user data after obtaining confirmation from the user regarding the prompt interface or pop-up window. Otherwise (i.e., if no confirmation is obtained from the user regarding the prompt interface or pop-up window), the relevant steps of collecting user data are terminated, and the user's relevant data is not collected.

[0224] In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use and processing of relevant user data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0225] The aforementioned user data includes information (including but not limited to user accounts), data (including but not limited to user-inputted text data, stored text / image data, displayed text / image data, etc.), and signal data. For example, the user data involved in this application was obtained with full authorization.

[0226] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0227] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An intelligent assessment and scoring method, characterized in that, The method includes: The management terminal displays the assessment indicator input interface, which includes a first assessment indicator setting item and a second assessment indicator setting item. Receive the structured assessment indicators set through the first assessment indicator setting item and the unstructured assessment indicators set through the second assessment indicator setting item sent by the management terminal; The user terminal displays a supporting materials upload interface, which includes supporting materials input fields. Receive multiple supporting documents sent by the user terminal and input through the supporting document input field; The titles of the multiple supporting materials are respectively input into a BERT-based title classification model to obtain the classification results of the multiple supporting materials output by the title classification model. Based on the classification results, the multiple supporting materials are divided into structured supporting materials and unstructured supporting materials. The unstructured supporting material is input into a BERTSUM-based text summarization extraction model to obtain the text summary of the unstructured supporting material output by the text summarization extraction model. The unstructured assessment indicators and the text summary are input into a BERT-based dual-tower text semantic matching model to obtain the matching probability between the unstructured assessment indicators and the text summary output by the dual-tower text semantic matching model. The score of the unstructured supporting material is obtained based on the matching probability. The structured supporting evidence is input into a BERT-based named entity recognition model to obtain the key data information of the structured supporting evidence output by the named entity recognition model. Obtain the first value corresponding to the first item in the structured assessment indicators; Obtain the second value corresponding to the first item in the key data information; The score of the structured supporting material is obtained based on the relationship between the first value and the second value; The scores of the unstructured testimony and the structured testimony are multiplied by a set weight and then added together to obtain the assessment scores of multiple pieces of supporting evidence.

2. The method according to claim 1, characterized in that, Before inputting the titles of the multiple supporting materials into the BERT-based title classification model, the following steps are also included: Obtain multiple sample titles of supporting materials, and a category label added to each sample title of supporting materials; the category label is used to indicate whether the supporting material corresponding to the sample title of supporting materials is structured supporting material or unstructured supporting material. The obtained supporting material title sample is input into the BERT-based title classification model to obtain a predicted probability distribution. The predicted probability distribution includes a first probability and a second probability. The first probability represents the probability that the supporting material sample corresponding to the supporting material title sample is structured supporting material, and the second probability represents the probability that the supporting material sample corresponding to the supporting material title sample is unstructured supporting material. Based on the predicted probability distribution and the probability of the category labels added to the title samples of the supporting materials, the first loss is calculated using the cross-entropy loss function; Based on the first loss, the parameters of the BERT-based title classification model are updated using the gradient descent algorithm; Repeat the above steps until the first loss converges to the preset condition.

3. The method according to claim 1, characterized in that, Before inputting the unstructured supporting material into the BERTSUM-based text summarization extraction model, the following steps are also included: The unstructured evidence material samples were preprocessed by adding CLS and SEP tags to each sentence in the unstructured evidence material samples. The preprocessed unstructured supporting material samples are input into a BERTSUM-based text summarization extraction model to obtain a predicted text summary. Based on the predicted text summary and the text summary corresponding to the unstructured supporting material sample, a second loss is calculated using a loss function; Based on the second loss, the parameters of the BERTSUM-based text summarization extraction model are updated using an optimization algorithm; Repeat the above steps until the second loss converges to the preset condition.

4. The method according to claim 1, characterized in that, Before inputting the unstructured evaluation metrics and the text summary into the BERT-based dual-tower text semantic matching model, the following steps are also included: Obtain unstructured assessment indicator samples, unstructured supporting material text summary samples corresponding to each unstructured assessment indicator sample, and matching tags. The matching tags are used to indicate the matching degree between the unstructured assessment indicator sample and its corresponding unstructured supporting material text summary sample. The unstructured assessment indicator sample and its corresponding unstructured supporting material text summary sample are input into the BERT-based text semantic matching dual-tower model to obtain a predicted matching degree. Based on the probability of the predicted matching degree and the probability of the matching label, a third loss is calculated using a loss function; Based on the third loss, the parameters of the BERT-based dual-tower text semantic matching model are updated using an optimization algorithm; Repeat the above steps until the third loss converges to the preset condition.

5. The method according to claim 1, characterized in that, Before inputting the structured supporting evidence into a BERT-based named entity recognition model and obtaining the key data information of the structured supporting evidence output by the named entity recognition model, the process further includes: Obtain structured supporting material samples and annotation information for each structured supporting material sample. The annotation information is used to indicate the key fields corresponding to the assessment indicators contained in the structured supporting material samples. The structured supporting material sample is input into the BERT-based named entity recognition model to obtain a predicted annotation information; Based on the predicted annotation information and the annotation information corresponding to the structured supporting material samples, the fourth loss is calculated using the log-likelihood loss function; Based on the fourth loss, the parameters of the BERT-based named entity recognition model are updated using an optimization algorithm; Repeat the above steps until the fourth loss converges to the preset condition.

6. The method according to claim 1, characterized in that, Based on the relationship between the first and second values, the score of the structured supporting evidence is obtained, including: In response to the first value being greater than the second value, a preset score is deducted from the score of the structured supporting material.

7. An intelligent assessment and scoring device, characterized in that, The device includes: The assessment indicator input interface display module is used to display the assessment indicator input interface through the management terminal. The assessment indicator input interface includes a first assessment indicator setting item and a second assessment indicator setting item. The performance indicator acquisition module is used to receive structured performance indicators set through the first performance indicator setting item and unstructured performance indicators set through the second performance indicator setting item sent by the management terminal. The supporting materials upload interface display module is used to display the supporting materials upload interface through the user terminal. The supporting materials upload interface includes supporting materials input items. The supporting materials acquisition module is used to receive multiple supporting materials sent by the user terminal and input through the supporting materials input field; The supporting material segmentation module is used to input the titles of multiple supporting materials into a BERT-based title classification model, obtain the classification results of each of the multiple supporting materials output by the title classification model, and classify the multiple supporting materials into structured supporting materials and unstructured supporting materials according to the classification results. The text summarization module is used to input the unstructured supporting material into a BERTSUM-based text summarization extraction model and obtain the text summary of the unstructured supporting material output by the text summarization extraction model. The first score acquisition module is used to input the unstructured assessment indicators and the text summary into a BERT-based text semantic matching dual-tower model, obtain the matching probability between the unstructured assessment indicators and the text summary output by the text semantic matching dual-tower model, and obtain the score of the unstructured supporting material based on the matching probability. The key data information acquisition module is used to input the structured supporting materials into a BERT-based named entity recognition model and acquire the key data information of the structured supporting materials output by the named entity recognition model. The first value acquisition module is used to acquire the first value corresponding to the first item in the structured assessment indicators; The second value acquisition module is used to acquire the second value corresponding to the first item in the key data information; The second score acquisition module is used to acquire the score of the structured supporting material based on the relationship between the first value and the second value. The final score acquisition module is used to multiply the scores of the unstructured testimony materials and the scores of the structured testimony materials by a set weight and add them together to obtain the assessment scores of multiple supporting evidence materials.

8. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, which is loaded and executed by the processor to implement the intelligent assessment and scoring method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the intelligent assessment and scoring method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; the computer instructions are read and executed by the processor of a computer device to implement the intelligent assessment and scoring method as described in any one of claims 1 to 6.

Images