Auxiliary scoring system with scoring consensus and method thereof

Abstract

An auxiliary scoring method with scoring consensus includes: providing a plurality of answer data and a plurality of corresponding score data as input data sets; inputting the input data sets into a neural network model to train the neural network model; validating the neural network model and evaluating the accuracy of the output prediction of the neural network model to establish an artificial intelligence model; using the artificial intelligence model to generate a plurality of attention maps and / or a plurality of feature maps; obtaining scoring consensus information based on the plurality of attention maps and / or a plurality of feature maps; and visualizing the scoring consensus information and using different colors to mark the consensus blocks and non-consensus blocks of the scoring consensus information.

Claims

1. An auxiliary scoring system with scoring consensus, comprising: a data preprocessing module configured to provide a plurality of answer data and a plurality of scoring data corresponding to the plurality of answer data as an input data set; an artificial intelligence model module configured to input the input data set into a neural network model for training to establish an artificial intelligence model, and to use the artificial intelligence model to generate a plurality of attention maps and a plurality of feature maps; a scoring consensus calculation module configured to obtain scoring consensus information based on the plurality of attention maps and / or the plurality of feature maps; and a computer vision module configured to visualize the scoring consensus information.

2. The auxiliary scoring system with scoring consensus as described in claim 1, wherein the data preprocessing module comprises: a data digitization module configured to digitize the answer data into a digitized answer data; and a data conversion module configured to convert the digitized answer data into vector data and the scoring data into annotation data, wherein a plurality of the vector data can form a vector data set and a plurality of the annotation data can form an annotation data set.

3. The auxiliary scoring system with scoring consensus as described in claim 1, wherein the artificial intelligence model module comprises: a training module for training the neural network model using a vector dataset and an annotation dataset as input datasets; a validation module for validating the neural network model using a five-fold cross-validation and evaluating the accuracy of one output prediction of the neural network model using a test dataset to establish the artificial intelligence model; and an output module for transmitting the plurality of attention maps and the plurality of feature maps generated by the artificial intelligence model to the scoring consensus calculation module, wherein the attention maps are generated by a transformer encoder of the artificial intelligence model understanding the input dataset.

4. The auxiliary scoring system with scoring consensus as described in claim 1, wherein the scoring consensus calculation module comprises: a union set calculation unit for calculating the union set of the plurality of attention graphs to obtain comprehensive information of the plurality of raters; an intersection set calculation unit for calculating the intersection of the union set of the plurality of attention graphs and each of the attention graphs to obtain consensus information of each rater on the comprehensive information; and a difference set calculation unit for calculating the difference set of the union set of the plurality of attention graphs and each of the attention graphs to obtain non-consensus information of each rater on the comprehensive information; wherein the scoring consensus information includes the comprehensive information, the consensus information, and the non-consensus information.

5. The auxiliary scoring system with scoring consensus as described in claim 1, wherein the scoring consensus calculation module comprises: a union set calculation unit for calculating the union set of the plurality of feature maps to obtain comprehensive information of the plurality of raters; an intersection set calculation unit for calculating the intersection of the union set of the plurality of feature maps and each of the feature maps to obtain consensus information of each rater on the comprehensive information; and a difference set calculation unit for calculating the difference set of the union set of the plurality of feature maps and each of the feature maps to obtain non-consensus information of each rater on the comprehensive information; wherein the scoring consensus information includes the comprehensive information, the consensus information, and the non-consensus information.

6. An auxiliary scoring system with scoring consensus as described in claim 1, wherein the computer vision module uses a computer vision library to visualize the scoring consensus information and uses different colors to distinguish consensus blocks from non-consensus blocks.

7. The auxiliary scoring system with scoring consensus as described in claim 1, wherein the artificial intelligence model module includes a scoring inference module, which uses the artificial intelligence model to infer a test taker's answer data based on the scoring consensus information to generate a plurality of features, a plurality of scores corresponding to the plurality of features, and a plurality of weights corresponding to the plurality of features, and calculates a suggested score based on the plurality of scores and the plurality of weights, wherein the plurality of features correspond to a plurality of scoring items of a scoring item.

8. The auxiliary scoring system with scoring consensus as described in claim 7, wherein the computer vision module is used to visualize the plurality of features, the plurality of scores corresponding to the plurality of features, and the plurality of weights corresponding to the plurality of features, and to distinguish the plurality of features using different colors.

9. An auxiliary scoring system with scoring consensus as described in claim 8, wherein the test subject's answer data is a dental model, the scoring item is aesthetics, and the plurality of scoring sub-items include equal gingival margins, smoothness of the abutment line angle and surface fineness, margin clarity and continuity, and the amount of tooth texture grinding on the occlusal surface.

10. An auxiliary scoring system with scoring consensus, comprising: a data digitization module configured to digitize a set of answer data into a digitized answer data set; a data conversion module configured to convert the digitized answer data into vector data and to convert a scoring data set corresponding to the answer data into annotation data, wherein a plurality of the vector data sets can form a vector data set and a plurality of the annotation data sets can form an annotation data set; a training module configured to use the vector data set and the annotation data set as an input data set to train a neural network model; a validation module configured to use a five-fold cross-validation to validate the neural network model and to evaluate the accuracy of an output prediction of the neural network model to establish an artificial intelligence model; an output module configured to output a plurality of attention maps and a plurality of feature maps generated by the artificial intelligence model; and a scoring consensus calculation module configured to calculate scoring consensus information based on the plurality of attention maps. A scoring inference module is configured to use an artificial intelligence model to infer a test taker's answer data based on the scoring consensus information to generate multiple features, multiple scores corresponding to the multiple features, and multiple weights, and to calculate a suggested score based on the multiple scores and multiple weights; and a computer vision module is configured to visualize the scoring consensus information, use different colors to distinguish consensus blocks from non-consensus blocks, and visualize the multiple features, the multiple scores, and the multiple weights, and use different colors to mark the multiple features.

11. An auxiliary scoring method with scoring consensus, comprising: (a) providing a plurality of answer data and a plurality of scoring data corresponding to the plurality of answer data as an input data set by a data preprocessing module; (b) inputting the input data set into a neural network model by an artificial intelligence model module to train the neural network model; (c) validating the neural network model by the artificial intelligence model module and evaluating the accuracy of one of the output predictions of the neural network model to establish an artificial intelligence model; (d) generating a plurality of attention maps and / or a plurality of feature maps by the artificial intelligence model by the artificial intelligence model by the artificial intelligence model; (e) obtaining scoring consensus information by a scoring consensus calculation module based on the plurality of attention maps and / or the plurality of feature maps; and (f) visualizing the scoring consensus information by a computer vision module and marking consensus blocks and non-consensus blocks with different colors.

12. The auxiliary scoring method with scoring consensus as described in claim 11, wherein step (a) comprises: digitizing the answer data into a digitized answer data; converting the digitized answer data into a vector data; and converting the scoring data corresponding to the answer data into annotation data, wherein a plurality of the vector data can form a vector data set, and a plurality of the annotation data can form an annotation data set.

13. The auxiliary scoring method with scoring consensus as described in claim 12, wherein step (b) comprises: using the vector dataset and the annotation dataset as the input dataset to train the artificial intelligence model.

14. The auxiliary scoring method with scoring consensus as described in claim 11, wherein step (c) comprises: validating the artificial intelligence model using a five-fold cross-validation; and evaluating the artificial intelligence model using a test dataset.

15. The auxiliary scoring method with scoring consensus as described in claim 11, wherein step (d) comprises: using a transformer encoder to understand the input dataset to generate the plurality of attention maps.

16. The auxiliary scoring method with scoring consensus as described in claim 11, wherein step (e) comprises: calculating the union of the plurality of attention graphs to obtain comprehensive information of one of the plurality of raters; calculating the intersection of the union of the plurality of attention graphs and each of the attention graphs to obtain consensus information of each rater on the comprehensive information; and calculating the difference between the union of the plurality of attention graphs and each of the attention graphs to obtain non-consensus information of each rater on the comprehensive information; wherein the scoring consensus information includes the comprehensive information, the consensus information, and the non-consensus information.

17. An auxiliary scoring method with scoring consensus as described in claim 11, wherein, Step (e) includes: calculating the union of the plurality of feature maps to obtain comprehensive information for one of the plurality of raters; calculating the intersection of the union of the plurality of feature maps and each of the feature maps to obtain consensus information for each of the raters on the comprehensive information; and calculating the difference between the union of the plurality of feature maps and each of the feature maps to obtain non-consensus information for each of the raters on the comprehensive information; wherein the rating consensus information includes the comprehensive information, the consensus information, and the non-consensus information.

18. An auxiliary scoring method with scoring consensus as described in claim 11, wherein step (f) comprises: visualizing the scoring consensus information using a computer vision library and using different colors to distinguish consensus blocks from non-consensus blocks.

19. The assisted scoring method with scoring consensus as described in claim 11, further comprising: (g) feeding back the scoring consensus information to the artificial intelligence model by the scoring consensus calculation module; (h) using the artificial intelligence model to infer a test taker's answer data based on the scoring consensus information to generate a plurality of features, a plurality of scores corresponding to the plurality of features, and a plurality of weights, and calculating a suggested score based on the plurality of scores and the plurality of weights, wherein the plurality of features correspond to a plurality of scoring items of a scoring item; and (i) visualizing the plurality of features, the plurality of scores, and the plurality of weights by the computer vision module, and marking the plurality of features with different colors.

20. An auxiliary scoring method with scoring consensus as described in claim 19, wherein the test subject's answer data is a dental model, the scoring item is aesthetics, and the plurality of scoring sub-items include equal gingival margins, smoothness and surface fineness of the abutment line angle, margin clarity and continuity, and the amount of tooth texture grinding on the occlusal surface.

Images