Intelligent assessment scoring method, device, equipment, medium and program product

Abstract

The invention provides an intelligent assessment scoring method and device, equipment, a medium and a program product, and relates to the technical field of computers. The method comprises the following steps: obtaining a classification result of an evidence material through a title classification model; obtaining a text abstract of the unstructured evidence material through a text abstract extraction model; through a text semantic matching double-tower model, obtaining a matching probability of the unstructured assessment index and the text abstract, and obtaining a score of the unstructured evidence material according to the matching probability; key data information of the structured evidence material is obtained through the named entity recognition model; obtaining a score of the structured evidence material according to a size relationship between a first numerical value and a second numerical value corresponding to a first item in the structured assessment index and the key data information; and respectively multiplying the score of the unstructured evidence material and the score of the structured evidence material by a set weight, and adding to obtain an assessment score.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer technology, and in particular to an intelligent assessment and scoring method, device, equipment, medium and program product. Background Art

[0002] With the continuous development of network technology, the application of deep learning technology has also received increasing attention. In work applications, it can utilize its intelligent features to assist staff in completing some assessment and scoring tasks.

[0003] In traditional intelligent assessment and scoring work, assessors need to read multiple pieces of supporting materials, and then calculate scores based on different assessment indicators in combination with the corresponding supporting materials.

[0004] However, due to the large number of uploaded supporting materials, assessors need to spend a lot of time and energy for review, which may lead to errors in the subsequent judgment of assessment scores, thus reducing the efficiency of the assessment work. Summary of the Invention

[0005] Embodiments of the present application provide an intelligent assessment and scoring method, device, equipment, medium and program product, which can improve the efficiency of product design. The technical solution is as follows:

[0006] On the one hand, an intelligent assessment and scoring method is provided, and the method includes:

[0007] Display an assessment indicator input interface through a management terminal, where the assessment indicator input interface includes a first assessment indicator setting item and a second assessment indicator setting item;

[0008] Receive the structured assessment indicators set through the first assessment indicator setting item and the unstructured assessment indicators set through the second assessment indicator setting item sent by the management terminal;

[0009] Display a supporting material upload interface through a user terminal, where the supporting material upload interface includes a supporting material input item;

[0010] Receive multiple supporting materials input through the supporting material input item sent by the user terminal;

[0011] Input the titles of the multiple supporting materials into a BERT-based title classification model respectively, obtain the classification results of each of the multiple supporting materials output by the title classification model, and divide the multiple supporting materials into structured supporting materials and unstructured supporting materials according to the classification results;

[0012] Input the unstructured supporting materials into the text summary extraction model based on BERT SUM to obtain the text summary of the unstructured supporting materials output by the text summary extraction model;

[0013] Input the unstructured assessment indicators and the text summary into the text semantic matching two-tower model based on BERT to obtain the matching probability between the unstructured assessment indicators and the text summary output by the text semantic matching two-tower model, and obtain the score of the unstructured supporting materials according to the matching probability;

[0014] Input the structured supporting materials into the named entity recognition model based on BERT to obtain the key data information of the structured supporting materials output by the named entity recognition model;

[0015] Obtain the first value corresponding to the first item in the structured assessment indicators;

[0016] Obtain the second value corresponding to the first item in the key data information;

[0017] Obtain the score of the structured supporting materials according to the magnitude relationship between the first value and the second value;

[0018] Multiply the scores of the unstructured supporting materials and the scores of the structured supporting materials by the set weight values respectively and add them to obtain the assessment scores of multiple supporting materials.

[0019] On the other hand, an intelligent assessment scoring device is provided, and the device includes:

[0020] An assessment indicator input interface display module, configured to display an assessment indicator input interface through a management terminal, where the assessment indicator input interface includes a first assessment indicator setting item and a second assessment indicator setting item;

[0021] An assessment indicator acquisition module, configured to receive the structured assessment indicators set through the first assessment indicator setting item and the unstructured assessment indicators set through the second assessment indicator setting item sent by the management terminal;

[0022] A supporting material upload interface display module, configured to display a supporting material upload interface through a user terminal, where the supporting material upload interface includes a supporting material input item;

[0023] A supporting material acquisition module, configured to receive multiple supporting materials input through the supporting material input item sent by the user terminal;

[0024] The supporting material classification module is used to input the titles of multiple said supporting materials into a BERT-based title classification model respectively, obtain the classification results of each of the multiple said supporting materials output by the title classification model, and divide the multiple said supporting materials into structured supporting materials and unstructured supporting materials according to the classification results;

[0025] The text abstract acquisition module is used to input the unstructured supporting materials into a text abstract extraction model based on BERTSUM, and obtain the text abstract of the unstructured supporting materials output by the text abstract extraction model;

[0026] The first score acquisition module is used to input the unstructured assessment indicators and the text abstract into a BERT-based text semantic matching dual tower model, obtain the matching probability of the unstructured assessment indicators and the text abstract output by the text semantic matching dual tower model, and obtain the score of the unstructured supporting materials according to the matching probability;

[0027] The key data information acquisition module is used to input the structured supporting materials into a BERT-based named entity recognition model, and obtain the key data information of the structured supporting materials output by the named entity recognition model;

[0028] The first value acquisition module is used to obtain the first value corresponding to the first item in the structured assessment indicators;

[0029] The second value acquisition module is used to obtain the second value corresponding to the first item in the key data information;

[0030] The second score acquisition module is used to obtain the score of the structured supporting materials according to the magnitude relationship between the first value and the second value;

[0031] The final score acquisition module is used to multiply the scores of the unstructured supporting materials and the structured supporting materials by set weight values respectively and add them up to obtain the assessment scores of the multiple said supporting materials.

[0032] In another aspect, a computer device is provided. The computer device includes a processor and a memory. At least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the above intelligent assessment scoring method.

[0033] In yet another aspect, a computer-readable storage medium is provided. At least one computer program is stored in the computer-readable storage medium, and the computer program is loaded and executed by a processor to implement the above intelligent assessment scoring method.

[0034] On the other hand, a computer program product is provided. The computer program product includes a computer program, which is stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium and executes the computer program, so that the computer device executes the intelligent assessment and scoring method provided in the above various optional implementation manners.

[0035] The technical solutions provided in the embodiments of the present application may include the following beneficial effects:

[0036] The computer device can display the assessment index input interface of the intelligent assessment system. After receiving the setting operations on the first assessment index setting item and the second assessment index setting item displayed in the assessment index input interface, the computer device obtains the structured assessment index and the unstructured assessment index. The computer device can display the supporting material upload interface of the intelligent assessment system. After receiving the operation on the supporting material input item displayed in the supporting material upload interface, the computer device obtains the structured supporting material and the unstructured supporting material. Then, through the background of the intelligent assessment system, the computer device respectively obtains the classification result of the supporting material, the matching probability between the unstructured assessment index and the text summary, and the key data information of the structured supporting material according to the title classification model, the text summary extraction model, the text semantic matching two-tower model, and the named entity recognition model. Then, the computer device obtains the score of the unstructured supporting material according to the matching probability, obtains the score of the structured supporting material according to the magnitude relationship between the first value corresponding to the first item in the structured assessment index and the second value corresponding to the first item in the key data information, and finally obtains the assessment score of the supporting material according to the score of the unstructured supporting material and the score of the structured supporting material. In the above solution, when it is necessary to assess and score the supporting materials, the staff only needs to set the parameters of the structured assessment index and the unstructured assessment index in the assessment index input interface of the management terminal. Subsequently, the server can automatically calculate the score according to different models in combination with the supporting materials uploaded by the user to obtain the assessment score. The above operation steps are easy to understand and operate, and do not require designers to invest a lot of energy in mastering and familiarizing with the assessment and scoring rules, which can greatly improve the assessment and scoring efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0038] Figure 2 is a flowchart of an intelligent assessment and scoring method provided by an embodiment of the present application;

[0039] Figure 3 is a flowchart of an intelligent assessment and scoring method provided by an embodiment of the present application;

[0040] Figure 4It is a flowchart of an intelligent assessment and scoring method provided by an embodiment of the present application;

[0041] Figure 5 It is a flowchart of an intelligent assessment and scoring method provided by an embodiment of the present application;

[0042] Figure 6 It is a flowchart of an intelligent assessment and scoring method provided by an embodiment of the present application;

[0043] Figure 7 It is a flowchart of an intelligent assessment and scoring method provided by an embodiment of the present application;

[0044] Figure 8 It is a flowchart of an intelligent assessment and scoring method based on deep learning technology involved in an embodiment of the present application;

[0045] Figure 9 It is a structural diagram of a title classification model based on BERT involved in an embodiment of the present application;

[0046] Figure 10 It is a structural diagram of a text summary extraction model based on BERTSUM involved in an embodiment of the present application;

[0047] Figure 11 It is a structural diagram of a text semantic matching two-tower model based on BERT involved in an embodiment of the present application;

[0048] Figure 12 It is a structural diagram of a named entity recognition model based on BERT involved in an embodiment of the present application;

[0049] Figure 13 It is a block diagram of an intelligent assessment and scoring device provided by an embodiment of the present application;

[0050] Figure 14 It is a structural block diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0051] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0052] 1) Supporting materials: Materials used to support or prove the authenticity and accuracy of other supporting materials, which can be data, information, documents, records, or other relevant materials. This material can be divided into structured supporting materials and unstructured supporting materials. Structured supporting materials mean that the supporting material can be quantified, and unstructured supporting materials mean that the supporting material cannot be quantified.

[0053] 2) Evaluation indicators: Criteria used to measure an individual's performance and achievements in a specific task or goal. These evaluation indicators can be divided into structured evaluation indicators and unstructured evaluation indicators. Structured evaluation indicators mean that the supporting materials can be quantified, while unstructured evaluation indicators mean that the evaluation indicators cannot be quantified.

[0054] 3) BERT (Bidirectional Encoder Representations from Transformers, a machine learning technology based on transformers) model: A natural language processing model whose main purpose is to provide in-depth semantic understanding and representation learning for text data.

[0055] 4) BERTSUM (BERT Extractive Summarizer) model: A text summarization tool based on the BERT model, used to automatically generate summaries or abstracts of text.

[0056] Figure 1 The figure shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application. The implementation environment may include: a first terminal 110, a second terminal 120, and a server 130.

[0057] In Figure 1 In the computer system 100, the first terminal 110 is the terminal used by the administrator to set evaluation indicators, and the second terminal 120 is the terminal used by the user to upload supporting materials.

[0058] The device types of the first terminal 120 and the second terminal 120 are the same or different, and the device types include at least one of a smart phone, a tablet computer, a laptop computer, and a desktop computer.

[0059] The first terminal 110 and the second terminal 120 are respectively connected to the server 130 through a wireless network or a wired network.

[0060] Please refer to Figure 2 , which shows a flowchart of an intelligent assessment and scoring method provided by an embodiment of the present application. For ease of explanation, only the server as the execution subject of each step is introduced. The server may be Figure 1 the server 130 shown; the method may include the following steps.

[0061] Step 200: Display an evaluation indicator input interface through the management terminal. The evaluation indicator input interface includes a first evaluation indicator setting item and a second evaluation indicator setting item.

[0062] Step 210: Receive the structured assessment indicators set through the first assessment indicator setting item and the unstructured assessment indicators set through the second assessment indicator setting item sent by the management terminal.

[0063] Step 220: Display a supporting material upload interface through the user terminal. The supporting material upload interface includes a supporting material input item.

[0064] Step 230: Receive multiple supporting materials input through the supporting material input item sent by the user terminal.

[0065] Step 240: Input the titles of multiple supporting materials into the BERT-based title classification model respectively, obtain the classification results of each of the multiple supporting materials output by the title classification model, and divide the multiple supporting materials into structured supporting materials and unstructured supporting materials according to the classification results.

[0066] Step 250: Input the unstructured supporting materials into the text summary extraction model based on BERTSUM, and obtain the text summary of the unstructured supporting materials output by the text summary extraction model.

[0067] Step 260: Input the unstructured assessment indicators and the text summary into the BERT-based text semantic matching two-tower model, obtain the matching probability between the unstructured assessment indicators and the text summary output by the text semantic matching two-tower model, and obtain the score of the unstructured supporting materials according to the matching probability.

[0068] Step 270: Input the structured supporting materials into the BERT-based named entity recognition model, and obtain the key data information of the structured supporting materials output by the named entity recognition model.

[0069] Step 280: Obtain the first value corresponding to the first item in the structured assessment indicators.

[0070] Step 290: Obtain the second value corresponding to the first item in the key data information.

[0071] Step 300: Obtain the score of the structured supporting materials according to the magnitude relationship between the first value and the second value.

[0072] Step 310: Multiply the scores of the unstructured supporting materials and the structured supporting materials by the set weight values respectively and add them to obtain the assessment scores of the multiple supporting materials.

[0073] In the solution shown in the embodiments of the present application, the computer device can display the assessment index input interface of the intelligent assessment system. After receiving the setting operations for the first assessment index setting item and the second assessment index setting item displayed in the assessment index input interface, the computer device obtains the structured assessment index and the unstructured assessment index. The computer device can display the supporting material upload interface of the intelligent assessment system. After receiving the operation on the supporting material input item displayed in the supporting material upload interface, the computer device obtains the structured supporting material and the unstructured supporting material. Then, through the background of the intelligent assessment system, the computer device respectively obtains the classification result of the supporting material, the matching probability between the unstructured assessment index and the text summary, and the key data information of the structured supporting material according to the title classification model, the text summary extraction model, the text semantic matching two-tower model, and the named entity recognition model. Then, the computer device obtains the score of the unstructured supporting material according to the matching probability, obtains the score of the structured supporting material according to the magnitude relationship between the first value corresponding to the first item in the structured assessment index and the second value corresponding to the first item in the key data information. Finally, the computer device obtains the assessment score of the supporting material according to the score of the unstructured supporting material and the score of the structured supporting material.

[0074] In summary, in the embodiments of the present application, when it is necessary to conduct an assessment and scoring of the supporting materials, the staff only needs to set the parameters of the structured assessment index and the unstructured assessment index in the assessment index input interface of the management terminal. Subsequently, the server can automatically calculate the score and obtain the assessment score according to different machine learning models in combination with the supporting materials uploaded by the user. The above operation steps are easy to understand and operate, and do not require designers to invest a lot of energy in mastering and familiarizing with the assessment and scoring rules, which can greatly improve the assessment and scoring efficiency.

[0075] Based on Figure 2 the solution shown, please refer to Figure 3 , which shows the flowchart of the intelligent assessment and scoring method provided by an embodiment of the present application. As Figure 3 shown, Figure 2 in the embodiment shown, before step 240, steps 231, 232, 233, and 234 may also be included.

[0076] Step 231: Obtain multiple supporting material title samples, and the category labels added to each supporting material title sample; the category labels are used to indicate whether the supporting material corresponding to the supporting material title sample is a structured supporting material or an unstructured supporting material.

[0077] The above category labels can be divided into meeting materials, work summaries, institutional mechanisms, etc., to ensure that the number of samples of each category in the training set is roughly the same.

[0078] Step 232: Input the obtained title samples of supporting materials into the BERT-based title classification model to obtain a predicted probability distribution, which includes a first probability and a second probability. The first probability represents the probability that the supporting material sample corresponding to the title sample of the supporting material is a structured supporting material, and the second probability represents the probability that the supporting material sample corresponding to the title sample of the supporting material is an unstructured supporting material.

[0079] The above BERT-based title classification model mainly consists of three modules: a data input module, a feature extraction module, and a classification module. The feature extraction module uses the BERT model to extract text features.

[0080] The server first inputs the title samples of the assessment supporting materials and their corresponding labels (x title , y cls ) into the input module. The input module encodes the title samples of the assessment supporting materials into low-dimensional space vectors. The encoding mainly consists of two parts: category and word vector encoding. In word vector encoding, the text content is tokenized to obtain tokens, and then the tokens are mapped to the corresponding token ids using the fixed vocabulary of the WordPiece model. Then, the encoded vectors are input into the feature extraction module for feature extraction to obtain the feature vector Y pred of the statement, where the feature extraction module needs to pre-load the weight parameters of the pre-trained BERT model.

[0081] Step 233: Calculate the first loss through the cross-entropy loss function based on the predicted probability distribution and the probability of the category label added to the title sample of the supporting material.

[0082] The Softmax function is part of the title classification model. Specifically, it can be the output layer in the title classification model. After the title classification model processes the title, it can output a probability distribution through the Softmax function. The formula of the Softmax function is shown in Formula 1:

[0083]

[0084] where C is the set of categories of the title samples of the assessment supporting materials, z i represents the output score of the current material model, z j represents the output scores of the current material in different categories, and P i represents the probability value of the i-th category in the probability distribution generated by the current material after passing through the Softmax function.

[0085] The cross-entropy loss function is used to measure the gap between the model output and the actual label. The formula of the cross-entropy loss function is shown in Formula 2 below:

[0086] L = -∑ i∈Cy i log(P i ) (2)

[0087] where y i represents the label value of the i-th category in the actual label, and L represents the loss value, which is used to measure the gap between the output probability value of the model and the actual label probability value. The smaller this gap is, the smaller the loss value, indicating that the predicted probability value of the model is closer to the probability value of the actual label.

[0088] Step 234: Update the parameters of the BERT-based title classification model based on the first loss through the gradient descent algorithm.

[0089] Repeat the above steps until the first loss converges to a preset condition.

[0090] In the embodiments of the present application, by combining the assessment supporting material title samples and the corresponding labels with the cross-entropy function to train the BERT-based title classification model, the noise of label assignment can be reduced, and the model can benefit from the training samples with correct labels, thereby improving the classification accuracy. On the other hand, the language understanding ability of BERT can be fully utilized to improve the performance of the model.

[0091] Based on Figure 2 or Figure 3 the scheme shown, please refer to Figure 4 , which shows the flowchart of the intelligent assessment scoring method provided by an embodiment of the present application. As Figure 4 shown, Figure 2 or Figure 3 in the embodiments shown, before step 250, steps 241, 242, 243, and 244 may also be included.

[0092] Step 241: Perform a preprocessing operation on the unstructured supporting material samples, and add CLS and SEP tags to each sentence in the unstructured supporting material samples respectively.

[0093] The above preprocessing operation can be implemented through code. CLS corresponds to the word vector of the first word in the input text, and SEP corresponds to the word vector of the last word in the input text. After adding CLS and SEP tags to each sentence in the unstructured supporting material samples respectively, it can be input into the model for training.

[0094] Step 242: Input the multiple unstructured supporting material samples after the preprocessing operation into the text summary extraction model based on BERTSUM to obtain a predicted text summary.

[0095] The above-mentioned text summarization extraction model based on BERT SUM mainly consists of three modules: a data input module, a feature extraction module, and a summary extraction module. Among them, the feature extraction module uses the BERT model to extract text features, and the summary extraction module adopts the transform + Sigmoid module.

[0096] First, the unstructured testimony material sample is input into the input module of the constructed BERT SUM text summarization extraction model, and the assessment testimony material is encoded into a low-dimensional space vector. Among them, the word vector encoding is to tokenize the text content to obtain tokens, and then use the fixed vocabulary of the WordPiece model to map the tokens to the corresponding token ids. Then, the encoded vector is input into the feature extraction module for feature extraction to obtain the feature vector of the sentence. The feature extraction module needs to pre-load the weight parameters of the pre-trained BERT model. Finally, the feature vector is input into the transform + Sigmoid module, and its formulas are shown in Formulas (3), (4), and (5):

[0097]

[0098]

[0099]

[0100] Among them, represents the hidden state value of the l-th layer obtained from the hidden state of the (L - 1)-th layer, h l-1 represents the hidden state value of the (L - 1)-th layer, h l represents the hidden state value of the L-th layer, represents the weight value, b o represents the bias value, σ represents the sigmoid function, represents the classification result, MHAtt represents the multi-head attention mechanism, and LN represents the regularization operation.

[0101] Step 243: Based on the predicted text summary and the text summary corresponding to the unstructured testimony material sample, calculate the second loss through the loss function.

[0102] Among them, the second loss can be calculated according to the difference between the predicted text summary and the text summary corresponding to the unstructured testimony material sample. For example, the semantic features of the predicted text summary and the text summary corresponding to the unstructured testimony material sample can be extracted, and then the semantic features of the predicted text summary and the text summary corresponding to the unstructured testimony material sample are input into the loss function, so as to calculate the second loss according to the difference between the semantic features of the predicted text summary and the text summary corresponding to the unstructured testimony material sample.

[0103] Step 244: Update the parameters of the text summarization extraction model based on BERT SUM through an optimization algorithm based on the second loss.

[0104] Repeat the above steps until the second loss converges to a preset condition.

[0105] In the embodiment of the present application, the text summarization extraction model based on BERT SUM is trained through unstructured testimonial material samples combined with a loss function. Unstructured testimonial material samples usually contain different text forms and contents, and the model can learn different text structures and word-using methods, improving the generalization ability of the model, thereby improving the efficiency of the model in the assessment scoring task.

[0106] Based on Figures 2 to 4 any of the shown solutions, please refer to Figure 5 , which shows a flowchart of an intelligent assessment scoring method provided by an embodiment of the present application. As Figure 5 shown, Figures 2 to 4 Before step 260 in any of the shown embodiments, steps 251, 252, 253, and 254 may further be included.

[0107] Step 251: Obtain unstructured assessment index samples, unstructured testimonial material text summary samples corresponding to each unstructured assessment index sample, and matching labels, where the matching labels are used to indicate the matching degree between the unstructured assessment index samples and their corresponding unstructured testimonial material text summary samples.

[0108] The above assessment index samples may be a piece of descriptive text, the text content being assessment key points and material submission requirements, and also including some added keywords according to the characteristics of special testimonial materials, such as important file names mentioned in the materials. The above text summary samples cover the main content of the testimonial materials.

[0109] Form the above assessment index samples and their corresponding text summary samples into matching text pairs, and add matching labels to each pair of matching text pairs. The matching labels may be matching or not matching. When the label is matching, it represents that the sample pair is a positive sample pair, and when the label is not matching, it represents that the sample pair is a negative sample pair, constructing a training set composed of positive and negative samples with a quantity ratio of 1:1.

[0110] Step 252: Input the unstructured assessment index samples and their corresponding unstructured testimonial material text summary samples into a two-tower model for text semantic matching based on BERT to obtain a predicted matching degree.

[0111] The above two-tower model for text semantic matching based on BERT mainly consists of three modules: a data input module, a feature extraction module, and a feature matching module, where the feature extraction module uses the BERT model to extract text features.

[0112] Input the above assessment index samples and their corresponding text abstract samples pairs (x cri , x sup ) into the input module respectively, and encode the text content information into low-dimensional space vectors. The encoding mainly consists of two parts: word vector encoding and position vector encoding. Among them, word vector encoding is to segment the text content to obtain tokens, and then use the fixed vocabulary of the WordPiece model to map the tokens to the corresponding token ids. The position vector encoding adopts the relative position encoding method, as shown in formula (6):

[0113]

[0114] Where is the word vector, R i-j is the relative position and i - j > 0, u T , v T and W are trainable weight matrices. Finally, perform a sum operation on the encoded word vector and position vector to obtain the final vector.

[0115] Then input the encoded vector into the feature extraction module to extract features to obtain the feature vectors of the sentences (y cri , y sup ). Among them, cri represents the feature vector of the assessment index, and sup represents the feature vector of the supporting materials.

[0116] The feature extraction module needs to pre-load the weight parameters of the pre-trained BERT model, and finally calculate the cosine matching degree of these two feature vectors, as shown in formula (7):

[0117]

[0118] Step 253: Calculate the third loss through the loss function based on the probability of the prediction matching degree and the probability of the matching label.

[0119] Use the cross-entropy loss function to calculate the loss of the model. The cross-entropy function formula is as shown in formula (8):

[0120]

[0121] Where γ is a smoothing coefficient of the cross-entropy function and is a hyperparameter. y sup is a set of supporting materials corresponding to an assessment index, including all positive samples and four random negative samples of the supporting materials. The cross-entropy loss function formula is as shown in formula (9):

[0122]

[0123] Where Represents the positive sample of the supporting material.

[0124] Step 254: Based on the third loss, update the parameters of the BERT-based text semantic matching two-tower model through an optimization algorithm.

[0125] Repeat the above steps until the third loss converges to a preset condition.

[0126] In the embodiment of the present application, the sample pair composed of the unstructured assessment index sample and its corresponding unstructured supporting material text summary sample combines with the cross-entropy loss function to train the BERT-based text semantic matching two-tower model. Because the BERT model has the ability of context awareness and can understand the context information in the text, through training based on BERT, the two-tower model can better understand the context of the assessment index and the supporting material text, so as to more accurately capture the semantic matching. On the other hand, the cross-entropy loss function usually has good numerical stability, which helps to avoid numerical problems such as gradient disappearance or gradient explosion, makes the training more reliable, and thus ensures the accuracy of the model.

[0127] Based on Figures 2 to 5 Any of the shown solutions, please refer to Figure 6 , which shows the flowchart of the intelligent assessment scoring method provided by an embodiment of the present application. As Figure 6 shown, Figures 2 to 6 In any of the shown embodiments, before step 270, steps 261, 262, 263, and 264 may also be included.

[0128] Step 261: Obtain structured supporting material samples and the annotation information of each structured supporting material sample. The annotation information is used to indicate the keyword fields corresponding to the assessment indexes included in the structured supporting material samples.

[0129] The above annotation information is specified to label the keywords of named entities according to specific requirements, such as time, place, number of meetings, etc.

[0130] Step 262: Input the structured supporting material samples into the BERT-based named entity recognition model to obtain a predicted annotation information.

[0131] The above BERT-based named entity recognition model mainly consists of three modules: a data input module, a feature extraction module, and a named entity recognition module. Among them, the feature extraction module uses the BERT model to extract text features, and the named entity recognition module adopts a linear chain conditional random field model.

[0132] First, the sample pair (x) composed of the above-mentioned supporting material samples and corresponding labels sup, (y) are respectively input into the input module, the text content information is encoded into a low-dimensional space vector, then the tokens are mapped to the corresponding token ids using the fixed vocabulary of the WordPiece model, and then the encoded vector is input into the feature extraction module for feature extraction to obtain the feature vector y of the statement. pred , where the feature extraction module needs to pre-load the weight parameters of the pre-trained BERT model. Finally, the feature vector y pred is input into the named entity module, that is, the feature vector is input into the conditional random field model. This model performs label prediction on each token to obtain a label sequence, and maps the label sequence back to the original text to obtain a predicted annotation information.

[0133] Step 263: Based on the predicted annotation information and the annotation information corresponding to the structured supporting material sample, calculate the fourth loss through the log-likelihood loss function.

[0134] In the embodiment of the present application, the fourth loss can be calculated through the log-likelihood loss function, that is, the loss of the model is calculated using the linear-chain conditional random field, as shown in Formulas (10) and (11):

[0135]

[0136]

[0137] Where and respectively represent the EmissionScore and Transition Score of the i-th label y i in the label sequence y corresponding to the supporting material.

[0138] Step 264: Based on the fourth loss, update the parameters of the BERT-based named entity recognition model through an optimization algorithm.

[0139] Repeat the above steps until the fourth loss converges to a preset condition.

[0140] In the embodiment of the present application, by training the BERT-based named entity recognition model using the sample pair composed of the supporting material sample and the corresponding label in combination with the log-likelihood loss function, the performance and generalization ability of the model can be improved, label noise can be reduced, context understanding can be enhanced, the risk of overfitting can be reduced, and the accuracy of the subsequent assessment and scoring task can be ensured.

[0141] Based on Figures 2 to 6 any of the above-described solutions, please refer to Figure 7 , which shows the flowchart of the intelligent assessment and scoring method provided by an embodiment of the present application. As Figure 7 shown, Figures 2 to 6In any of the illustrated embodiments, step 300 can be implemented as step 300a.

[0142] Step 300a: In response to the first value being greater than the second value, a preset score is deducted from the score of the structured supporting material.

[0143] The first value is the value corresponding to the first item in the structured assessment indicator, and the second value is the value corresponding to the first item in the key data information. Compare the magnitudes of the two values. If the first value is greater than the second value, a preset score is deducted from the score of the structured supporting material.

[0144] For example, if the structured assessment indicator requires 5 meetings this year, but it is analyzed from the key data information of the materials that only 4 meetings were held, a certain score is deducted according to the assessment rules.

[0145] In the embodiments of the present application, a preset score is deducted from the score of the structured supporting material according to the numerical comparison, making the calculation of the assessment score more convenient and improving the efficiency of the assessment scoring.

[0146] Combined with the above Figures 2 to 7 illustrated solution, taking the intelligent assessment scoring method based on deep learning technology as an example, the specific embodiments will be described in detail. It should be understood that the specific implementation manners described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application.

[0147] Figure 8 is a flowchart of an intelligent assessment scoring method based on deep learning technology according to an embodiment of the present application. The method includes:

[0148] Step 1: The assessment indicators are manually divided into structured indicators and unstructured indicators, and the supporting materials are input into the title classification model to automatically divide them into structured supporting materials and unstructured supporting materials;

[0149] Step 2: For the unstructured supporting materials, they are input into the text summarization model to obtain the text summary of the supporting materials, and the text summary can also be used as a reference for the inspectors;

[0150] Step 3: The unstructured indicators and the text summary of the supporting materials in step 2 are respectively input into the twin tower model to obtain the matching probability between the unstructured indicators and the text summary of the supporting materials and use it as the score of the unstructured supporting materials;

[0151] Step 4: For the structured supporting materials, they are input into the information extraction model to extract the key data information in the supporting materials;

[0152] Step 5: Compare the key data information of the supporting materials in Step 4 with the structured indicators and perform statistical analysis to calculate the score of the structured supporting materials;

[0153] Step 6: Multiply the scores of the unstructured supporting materials in Step 3 and the scores of the structured supporting materials in Step 5 by the set weight values and add them to obtain the final score of this assessment indicator.

[0154] Figure 9 It is a structural diagram of a BERT-based title classification model involved in an embodiment of the application. The title classification model of the supporting materials is constructed in the following way:

[0155] Step 1: Collect the titles of the assessment supporting materials and add category labels to each title. The labels of the supporting materials are divided into meeting materials, work summaries, institutional mechanisms, etc. Divide the constructed dataset into a training set, a validation set, and a test set. To reduce the influence of the training set on the model, try to ensure that the number of samples of each category in the training set is roughly the same.

[0156] Step 2: Construct a BERT-based title classification model, which mainly consists of three modules: a data input module, a feature extraction module, and a classification module. Among them, the feature extraction module uses the BERT model to extract text features.

[0157] Step 3: Train the BERT-based title classification model constructed in Step 2 on the training set in Step 1. First, input the titles of the assessment supporting materials and the corresponding labels into the input module. The main function of the input module is to encode the titles of the assessment supporting materials into low-dimensional space vectors. The encoding mainly consists of two parts: category and word vector encoding. Among them, the word vector encoding is to tokenize the text content to obtain tokens, and then use the fixed vocabulary of the WordPiece model to map the tokens to the corresponding token ids; then input the encoded vector into the feature extraction module to extract the feature vector of the statement. The feature extraction module needs to pre-load the weight parameters of the pre-trained BERT model in advance; finally, use the cross-entropy loss function to calculate the loss of the model;

[0158] Step 4: Input the titles and labels of the supporting materials in the validation set into the model trained in Step 3 to obtain the classification results of the supporting materials.

[0159] Figure 10 It is a structural diagram of a text summary extraction model based on BERTSUM involved in an embodiment of the application. The text summary extraction model of the supporting materials is constructed in the following way:

[0160] Step 1: Perform preprocessing operations on the unstructured supporting materials, and add CLS and SEP markers before and after each sentence in the materials;

[0161] Step 2: Construct a BERT SUM text summarization extraction model, which mainly consists of three modules: a data input module, a feature extraction module, and a summarization extraction module. Among them, the feature extraction module uses the BERT model to extract text features, and the summarization extraction module adopts a transform + Sigmoid module;

[0162] Step 3: Input the preprocessed assessment supporting materials in Step 1 into the input module of the BERT SUM text summarization extraction model constructed in Step 2. The main function of the input module is to encode the assessment supporting materials into low-dimensional space vectors. Among them, word vector encoding is to tokenize the text content to obtain tokens, and then use the fixed vocabulary of the WordPiece model to map the tokens to the corresponding token ids; then input the encoded vectors into the feature extraction module for feature extraction to obtain the feature vectors of the sentences. The feature extraction module needs to pre-load the weight parameters of the pre-trained BERT model in advance; finally, the feature vectors are input into the transform + Sigmoid module;

[0163] Step 4: Sort the results of Step 3 and select the top K sentences as the text summary of the assessment supporting materials.

[0164] Figure 11 It is a structural diagram of a two-tower model for text semantic matching based on BERT involved in an embodiment of the present application. The two-tower model for text semantic matching between the assessment indicators and the supporting materials is constructed in the following manner:

[0165] Step 1: Collect unstructured assessment indicators and their corresponding unstructured assessment supporting material summaries. The assessment indicators are mostly a paragraph of descriptive text, and the main content is the assessment key points and material submission requirements. At the same time, keywords are added according to the characteristics of some special supporting materials, such as important file names mentioned in the materials; the assessment supporting material summaries are mainly the condensation and refinement of the supporting materials, covering the main content of the supporting materials. Combine the assessment indicators and their corresponding supporting material summaries into matching text pairs, and add matching or non-matching labels to each pair of matching text pairs, which means that not only positive sample pairs need to be constructed, but also negative sample pairs need to be constructed. In order to reduce the influence of the training set on the model, try to ensure that the number of positive samples and negative samples in the training set is 1:1. Finally, divide the data set into a training set, a validation set, and a test set.

[0166] Step 2: Construct a two-tower model for text semantic matching based on BERT, which mainly consists of three modules: a data input module, a feature extraction module, and a feature matching module. Among them, the feature extraction module uses the BERT model to extract text features.

[0167] Step 3: Train the BERT-based text semantic matching two-tower model constructed in Step 2 on the training set in Step 1. First, input the assessment indicators and the summary pairs of supporting materials into the input module respectively. The main function of the input module is to encode the text content information in Step 1 into low-dimensional space vectors. The encoding mainly consists of two parts: word vector encoding and position vector encoding. Among them, word vector encoding is to segment the text content to obtain tokens, and then use the fixed vocabulary of the WordPiece model to map the tokens to the corresponding token ids; while position vector encoding adopts the relative position encoding method. Finally, perform an addition operation on the encoded word vectors and position vectors to obtain the final vector. Then input the encoded vectors into the feature extraction module to extract the feature vectors of the sentences. The feature extraction module needs to pre-load the weight parameters of the pre-trained BERT model and calculate the cosine matching degree of these two feature vectors. Finally, use the cross-entropy loss function to calculate the loss of the model.

[0168] Step 4: Input the assessment indicators and all the summary pairs of supporting materials in the validation set into the model trained in Step 3 in sequence to obtain the matching probability of the summary pairs of supporting materials and use it as the score of the unstructured assessment indicators.

[0169] Figure 12 It is the structure diagram of the BERT-based named entity recognition model involved in an embodiment of the present application. The key index field value recognition model in the supporting materials is constructed in the following way:

[0170] Step 1: Collect structured supporting materials and label them according to the key fields in the structured assessment indicators, and divide the constructed dataset into a training set, a validation set, and a test set;

[0171] Step 2: Construct a BERT-based named entity recognition model, which mainly consists of three modules: a data input module, a feature extraction module, and a named entity recognition module. Among them, the feature extraction module uses the BERT model to extract text features, and the named entity recognition module adopts a linear chain conditional random field module;

[0172] Step 3: Train the BERT-based named entity recognition model in Step 2 on the training set in Step 1. First, input the assessment supporting materials and corresponding labels into the input module respectively. The main function of the input module is to encode the text content information in Step 1 into low-dimensional space vectors. The encoding mainly consists of two parts: category and word vector encoding. Among them, word vector encoding is to segment the text content to obtain tokens, and then use the fixed vocabulary of the WordPiece model to map the tokens to the corresponding token ids. Then, input the encoded vectors into the feature extraction module to extract the feature vectors of the statements. The feature extraction module needs to pre-load the weight parameters of the pre-trained BERT model in advance. Finally, use the linear-chain conditional random field to calculate the loss of the model;

[0173] Step 4: Statistically analyze the keyword field values of the obtained supporting materials and the assessment indicators to obtain the scores of the structured indicators.

[0174] In the embodiment of the present application, an intelligent assessment and scoring method based on deep learning technology is provided to realize intelligent analysis of audit materials through artificial intelligence methods, so as to reduce the time and energy of manual review of supporting materials. Compared with the traditional manual review of materials and scoring, the review time is greatly reduced, and the work efficiency of office staff is improved.

[0175] The following is the device embodiment of the present application, which can be used to execute the method embodiment of the present application. For the details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0176] Please refer to Figure 13 , which shows the block diagram of the intelligent assessment and scoring device provided by an embodiment of the present application. The device may include the following modules:

[0177] The assessment index input interface display module 1300 is used to display the assessment index input interface through the management terminal. The assessment index input interface includes a first assessment index setting item and a second assessment index setting item;

[0178] The assessment index acquisition module 1301 is used to receive the structured assessment index set through the first assessment index setting item and the unstructured assessment index set through the second assessment index setting item sent by the management terminal;

[0179] The supporting material upload interface display module 1302 is used to display the supporting material upload interface through the user terminal. The supporting material upload interface includes a supporting material input item;

[0180] The supporting material acquisition module 1303 is used to receive multiple supporting materials input through the supporting material input item sent by the user terminal;

[0181] The supporting material classification module 1304 is configured to input the titles of multiple supporting materials into the BERT-based title classification model respectively, obtain the classification results of each of the multiple supporting materials output by the title classification model, and divide the multiple supporting materials into structured supporting materials and unstructured supporting materials according to the classification results;

[0182] The text summary obtaining module 1305 is configured to input the unstructured supporting materials into the text summary extraction model based on BERTSUM, and obtain the text summary of the unstructured supporting materials output by the text summary extraction model;

[0183] The first score obtaining module 1306 is configured to input the unstructured assessment indicators and the text summary into the BERT-based text semantic matching two-tower model, obtain the matching probability between the unstructured assessment indicators and the text summary output by the text semantic matching two-tower model, and obtain the score of the unstructured supporting materials according to the matching probability;

[0184] The key data information obtaining module 1307 is configured to input the structured supporting materials into the BERT-based named entity recognition model, and obtain the key data information of the structured supporting materials output by the named entity recognition model;

[0185] The first value obtaining module 1308 is configured to obtain the first value corresponding to the first item in the structured assessment indicators;

[0186] The second value obtaining module 1309 is configured to obtain the second value corresponding to the first item in the key data information;

[0187] The second score obtaining module 1310 is configured to obtain the score of the structured supporting materials according to the magnitude relationship between the first value and the second value;

[0188] The final score obtaining module 1311 is configured to multiply the scores of the unstructured supporting materials and the structured supporting materials by the set weights respectively and add them up to obtain the assessment scores of the multiple supporting materials.

[0189] In some embodiments, the above device further includes a title classification model training module for,

[0190] obtaining multiple supporting material title samples, and the category labels added to each supporting material title sample; the category labels are used to indicate whether the supporting materials corresponding to the supporting material title samples are structured supporting materials or unstructured supporting materials;

[0191] Input the obtained title samples of supporting materials into the BERT-based title classification model to obtain a predicted probability distribution, which includes a first probability and a second probability. The first probability represents the probability that the supporting material sample corresponding to the title sample of the supporting material is a structured supporting material, and the second probability represents the probability that the supporting material sample corresponding to the title sample of the supporting material is an unstructured supporting material;

[0192] Calculate the first loss through the cross-entropy loss function based on the predicted probability distribution and the probability of the category label added to the title sample of the supporting material;

[0193] Update the parameters of the BERT-based title classification model through the gradient descent algorithm based on the first loss;

[0194] Repeat the above steps until the first loss converges to a preset condition.

[0195] In some embodiments, the above device further includes a text summary extraction model training module for,

[0196] Perform preprocessing operations on the unstructured supporting material samples, and add CLS and SEP tags to each sentence in the unstructured supporting material samples respectively;

[0197] Input the multiple preprocessed unstructured supporting material samples into the BERTSUM-based text summary extraction model to obtain a predicted text summary;

[0198] Calculate the second loss through the loss function based on the predicted text summary and the text summary corresponding to the unstructured supporting material sample;

[0199] Update the parameters of the BERTSUM-based text summary extraction model through the optimization algorithm based on the second loss;

[0200] Repeat the above steps until the second loss converges to a preset condition.

[0201] In some embodiments, the above device further includes a text semantic matching two-tower model training module for,

[0202] Obtain unstructured assessment index samples, unstructured supporting material text summary samples corresponding to each unstructured assessment index sample, and matching labels. The matching labels are used to indicate the matching degree between the unstructured assessment index sample and its corresponding unstructured supporting material text summary sample;

[0203] Input the unstructured assessment index sample and its corresponding unstructured supporting material text summary sample into the BERT-based text semantic matching two-tower model to obtain a predicted matching degree;

[0204] Calculate the third loss through a loss function based on the probability of prediction matching degree and the probability of matching labels;

[0205] Based on the third loss, update the parameters of the dual - tower model for text semantic matching based on BERT through an optimization algorithm;

[0206] Repeat the above steps until the third loss converges to a preset condition.

[0207] In some embodiments, the above - mentioned device further includes a named - entity recognition model training module, which is used to,

[0208] Obtain structured supporting material samples and the annotation information of each structured supporting material sample. The annotation information is used to indicate the keyword fields corresponding to the assessment indicators included in the structured supporting material sample;

[0209] Input the structured supporting material samples into the named - entity recognition model based on BERT to obtain a predicted annotation information;

[0210] Based on the predicted annotation information and the annotation information corresponding to the structured supporting material samples, calculate the fourth loss through a logarithmic likelihood loss function;

[0211] Based on the fourth loss, update the parameters of the named - entity recognition model based on BERT through an optimization algorithm;

[0212] Repeat the above steps until the fourth loss converges to a preset condition.

[0213] In some embodiments, the second score acquisition module 1310 is specifically used to,

[0214] Deduct a preset score in response to the first value being greater than the second value.

[0215] Figure 14 FIG. shows the structural block diagram of a computer device 1400 provided by an exemplary embodiment of the present application. For example: personal computers, smart phones, and tablet computers. The computer device 1400 may also be referred to by other names such as user equipment.

[0216] Generally, the computer device 1400 includes: a processor 1401 and a memory 1402.

[0217] The memory 1402 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1402 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices and flash storage devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1402 is used to store at least one instruction for being executed by the processor 1401 to implement all or part of the steps performed by the computer-aided design software tool in the methods provided in the embodiments of the present application.

[0218] In some embodiments, the computer device 1400 may further optionally include: a peripheral device interface 1403 and at least one peripheral device. Specifically, the peripheral device includes at least one of a radio frequency circuit 1404, a touch display screen 1405, a camera 1406, an audio circuit 1407, and a power supply 1408.

[0219] In some embodiments, the computer device 1400 further includes one or more sensors 1409. The one or more sensors 1409 include, but are not limited to, an acceleration sensor 1410, a gyroscope sensor 1411, a pressure sensor 1412, an optical sensor 1413, and a proximity sensor 1414.

[0220] Those skilled in the art can understand that the structures shown above do not constitute a limitation on the computer device 1400, and it may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0221] In an exemplary embodiment, there is also provided a computer-readable storage medium for storing at least one computer program, and the at least one computer program is loaded and executed by a processor to implement all or part of the steps in the methods shown in the above various embodiments. For example, the computer-readable storage medium may be a read-only memory, a random access memory, a compact disc read-only memory, magnetic tape, a floppy disk, and an optical data storage device, etc.

[0222] In an exemplary embodiment, there is also provided a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of the computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes all or part of the steps in the methods shown in the above various embodiments.

[0223] Before collecting relevant data of the user and during the process of collecting relevant data of the user, this application can display a prompt interface, a pop-up window or output a voice prompt message, which is used to prompt the user that relevant data of the user is being collected currently. So that this application only starts to execute the relevant steps of obtaining relevant data of the user after obtaining the confirmation operation of the user on the prompt interface or the pop-up window. Otherwise (that is, when the confirmation operation of the user on the prompt interface or the pop-up window is not obtained), the relevant steps of obtaining relevant data of the user are ended, that is, relevant data of the user is not obtained.

[0224] In other words, all user data collected by this application is collected with the consent and authorization of the user, and the collection, use and processing of relevant user data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0225] The relevant data of the above-mentioned user includes information (including but not limited to the user's account), data (including but not limited to text data input by the user, stored text / image data, displayed text / image data, etc.) and signals and other data. For example, the user data involved in this application is obtained under sufficient authorization.

[0226] It should be understood that "a plurality of" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. These three situations. The character " / " generally represents an "or" relationship between the front and rear associated objects.

[0227] The above are only exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

1. An intelligent assessment and scoring method, characterized in that, the method includes: displaying an assessment index input interface through a management terminal, where the assessment index input interface includes a first assessment index setting item and a second assessment index setting item; receiving the structured assessment index set through the first assessment index setting item and the unstructured assessment index set through the second assessment index setting item sent by the management terminal; displaying a supporting material upload interface through a user terminal, where the supporting material upload interface includes a supporting material input item; receiving multiple supporting materials input through the supporting material input item sent by the user terminal; inputting the titles of the multiple supporting materials into a BERT-based title classification model respectively, obtaining the classification results of each of the multiple supporting materials output by the title classification model, and dividing the multiple supporting materials into structured supporting materials and unstructured supporting materials according to the classification results; inputting the unstructured supporting materials into a text summary extraction model based on BERTSUM, and obtaining the text summary of the unstructured supporting materials output by the text summary extraction model; inputting the unstructured assessment index and the text summary into a text semantic matching two-tower model based on BERT, obtaining the matching probability of the unstructured assessment index and the text summary output by the text semantic matching two-tower model, and obtaining the score of the unstructured supporting materials according to the matching probability; inputting the structured supporting materials into a BERT-based named entity recognition model, and obtaining the key data information of the structured supporting materials output by the named entity recognition model; obtaining a first value corresponding to a first item in the structured assessment index; obtaining a second value corresponding to the first item in the key data information; obtaining the score of the structured supporting materials according to the magnitude relationship between the first value and the second value; multiplying the scores of the unstructured supporting materials and the scores of the structured supporting materials by set weight values respectively and adding them to obtain the assessment scores of the multiple supporting materials.

2. The method according to claim 1, characterized in that, before inputting the titles of the multiple supporting materials into the BERT-based title classification model, it further includes: obtaining multiple supporting material title samples, and category labels added to each of the supporting material title samples; the category labels are used to indicate whether the supporting materials corresponding to the supporting material title samples are structured supporting materials or unstructured supporting materials; inputting the obtained supporting material title samples into the BERT-based title classification model, obtaining a predicted probability distribution, the predicted probability distribution including a first probability and a second probability, the first probability indicating the probability that the supporting material sample corresponding to the supporting material title sample is a structured supporting material, and the second probability indicating the probability that the supporting material sample corresponding to the supporting material title sample is an unstructured supporting material; Calculate the first loss through the cross - entropy loss function based on the predicted probability distribution and the probability of the category label added to the title sample of the supporting material. Update the parameters of the BERT - based title classification model through the gradient descent algorithm based on the first loss. Repeat the above steps until the first loss converges to a preset condition.

3. The method according to claim 1, wherein, before inputting the unstructured supporting material into the text summary extraction model based on BERTSUM, it further includes: Perform pre - processing operations on the unstructured supporting material samples, and add CLS and SEP tags to each sentence in the unstructured supporting material samples respectively. Input the multiple unstructured supporting material samples after the pre - processing operation into the text summary extraction model based on BERTSUM to obtain a predicted text summary. Calculate the second loss through the loss function based on the predicted text summary and the text summary corresponding to the unstructured supporting material sample. Update the parameters of the text summary extraction model based on BERTSUM through the optimization algorithm based on the second loss. Repeat the above steps until the second loss converges to the preset condition.

4. The method according to claim 1, wherein, before inputting the unstructured assessment indicators and the text summary into the BERT - based text semantic matching two - tower model, it further includes: Obtain unstructured assessment indicator samples, the text summary samples of the unstructured supporting materials corresponding to each unstructured assessment indicator sample, and matching labels, where the matching labels are used to indicate the matching degree between the unstructured assessment indicator sample and its corresponding text summary sample of the unstructured supporting material. Input the unstructured assessment indicator sample and its corresponding text summary sample of the unstructured supporting material into the BERT - based text semantic matching two - tower model to obtain a predicted matching degree. Calculate the third loss through the loss function based on the probability of the predicted matching degree and the probability of the matching label. Update the parameters of the BERT - based text semantic matching two - tower model through the optimization algorithm based on the third loss. Repeat the above steps until the third loss converges to the preset condition.

5. The method according to claim 1, wherein, before the server extracts the key data information of the structured supporting material through the BERT - based named entity recognition model, it further includes: Obtain structured supporting material samples and the annotation information of each structured supporting material sample, where the annotation information is used to indicate the keyword fields corresponding to the assessment indicators included in the structured supporting material sample. Input the structured supporting material sample into the BERT - based named entity recognition model to obtain a predicted annotation information. Calculate the fourth loss through the log - likelihood loss function based on the predicted annotation information and the annotation information corresponding to the structured supporting material sample. Update the parameters of the BERT - based named entity recognition model through the optimization algorithm based on the fourth loss. Repeat the above steps until the fourth loss converges to the preset condition.

6. The method according to claim 1, wherein, obtaining a score for the structured supporting material according to the magnitude relationship between the first value and the second value, including: in response to the first value being greater than the second value, deducting a preset score from the score of the structured supporting material.

7. An intelligent assessment and scoring device, wherein, the device includes: an assessment index input interface display module, configured to display an assessment index input interface through a management terminal, and the assessment index input interface includes a first assessment index setting item and a second assessment index setting item; an assessment index obtaining module, configured to receive the structured assessment index set through the first assessment index setting item and the unstructured assessment index set through the second assessment index setting item sent by the management terminal; a supporting material upload interface display module, configured to display a supporting material upload interface through a user terminal, and the supporting material upload interface includes a supporting material input item; a supporting material obtaining module, configured to receive a plurality of supporting materials input through the supporting material input item sent by the user terminal; a supporting material classification module, configured to respectively input the titles of the plurality of supporting materials into a BERT-based title classification model, obtain the classification results of the plurality of supporting materials output by the title classification model, and classify the plurality of supporting materials into structured supporting materials and unstructured supporting materials according to the classification results; a text summary obtaining module, configured to input the unstructured supporting material into a BERTSUM-based text summary extraction model, and obtain the text summary of the unstructured supporting material output by the text summary extraction model; a first score obtaining module, configured to input the unstructured assessment index and the text summary into a BERT-based text semantic matching two-tower model, obtain the matching probability between the unstructured assessment index and the text summary output by the text semantic matching two-tower model, and obtain the score of the unstructured supporting material according to the matching probability; a key data information obtaining module, configured to input the structured supporting material into a BERT-based named entity recognition model, and obtain the key data information of the structured supporting material output by the named entity recognition model; a first value obtaining module, configured to obtain a first value corresponding to a first item in the structured assessment index; a second value obtaining module, configured to obtain a second value corresponding to the first item in the key data information; a second score obtaining module, configured to obtain the score of the structured supporting material according to the magnitude relationship between the first value and the second value; a final score obtaining module, configured to multiply the scores of the unstructured supporting material and the structured supporting material by set weights respectively and add them up to obtain the assessment scores of the plurality of supporting materials.

8. A computer device, wherein, The computer device includes a processor and a memory, and the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the intelligent assessment and scoring method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores at least one computer program, and the computer program is loaded and executed by a processor to implement the intelligent assessment and scoring method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, the computer program product includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; the computer instructions are read and executed by a processor of a computer device to implement the intelligent assessment and scoring method as described in any one of claims 1 to 6.

Images