Review system and method based on adaptive scoring algorithm and multi-modal analysis

Abstract

The invention relates to a review system and method based on an adaptive scoring algorithm and multi-modal analysis, in particular to the field of college student innovation and entrepreneurship competition review, and the system specifically comprises a data input processing module, an automatic scoring calculation module, an expert review simulation module and an auxiliary decision support module. According to the technical scheme, the scoring weight and the standard can be dynamically adjusted according to the specific requirements and the standard of the contest, through the machine learning technology, historical review data are analyzed, key review factors are identified, the scoring model is automatically adjusted according to the characteristics and the target of the current contest, and the self-adaptive capability enables the review system to more accurately evaluate the competing works, so that the quality of the competing works is improved. And meanwhile, human prejudice is reduced.

Description

Technical Field

[0001] The present invention relates to the field of evaluation of college students' innovation and entrepreneurship competitions. More specifically, the present invention relates to an evaluation system and method based on an adaptive scoring algorithm and multimodal analysis. Background Art

[0002] As an important platform to promote innovation and entrepreneurship education, the college students' innovation and entrepreneurship competition aims to stimulate the entrepreneurial enthusiasm of college students and cultivate their innovative spirit and practical ability. The scoring system plays a crucial role in the competition. It not only affects the evaluation results but also provides important feedback to the contestants. However, traditional scoring methods usually rely on manual evaluation, and the evaluation process is easily affected by subjective factors, lacking transparency and consistency. Therefore, we propose an evaluation system and method based on an adaptive scoring algorithm and multimodal analysis. Summary of the Invention

[0003] In view of the technical problems existing in the prior art, the present invention provides an evaluation system and method based on an adaptive scoring algorithm and multimodal analysis. Through machine learning technology, historical evaluation data is analyzed to identify key evaluation factors, and the scoring model is automatically adjusted according to the characteristics and objectives of the current competition. This adaptive ability enables the evaluation system to more accurately evaluate the participating works and reduce human bias, so as to solve the problems raised in the above background art.

[0004] To achieve the above object, the present invention provides the following technical solution, an evaluation system based on an adaptive scoring algorithm and multimodal analysis: specifically including: a data input processing module, an automatic scoring calculation module, an expert evaluation simulation module, and an auxiliary decision-making support module;

[0005] Data Input Processing Module: Obtain the text and video participating work data of the college students' innovation and entrepreneurship competition, and use the TF-IDF of the natural language processing NLP algorithm to extract the key information in the text participating work data, use video analysis technology to extract the metadata and key visual information of the video content, fuse the text feature vectors extracted by NLP and the visual feature vectors extracted by video analysis by means of mid-term fusion of the multimodal BERT learning model, and use the shared Transformer structure to jointly represent the features of the text and video, so that the model can capture the correlation between the two modalities, and transmit the fused features to the automatic scoring calculation module for scoring, providing the required input for subsequent scoring;

[0006] Automatic Scoring Calculation Module: After obtaining the fused feature data transmitted by the data input processing module, based on the preset scoring criteria, automatically perform a preliminary scoring result for each participating work;

[0007] Expert review simulation module: By organizing the basic principles and historical data of expert scoring, a knowledge base is constructed. Then, based on the information in the knowledge base, simulated expert scores are generated, creating an expert interaction interface to display the automatic scoring results and allowing experts to adjust and modify the scores. All decision-making processes will be recorded, including the basis, data, and model parameters, ensuring the fairness and transparency of the scoring. As expert feedback is continuously incorporated, the scoring criteria are continuously improved, guaranteeing the accuracy and fairness of the review results. At the same time, expert feedback is collected, and reinforcement learning is used to adjust the scoring model and optimize the review process. The scoring model and criteria are continuously optimized according to expert feedback, continuously improving the quality and effectiveness of the review process;

[0008] Auxiliary decision-making support module: In the expert interaction interface, data analysis and visualization tools are provided for experts, and multi-objective optimization algorithms are used to help experts make optimized decisions according to the requirements of each scoring dimension, ensuring the comprehensiveness and accuracy of the review.

[0009] In a preferred embodiment, the TF-IDF calculation formula of the natural language processing NLP algorithm in the data input processing module is:

[0010]

[0011] Where t represents a word, d represents a document, tf(t, d) represents the term frequency of word t in document d, df(t) represents the number of documents containing word t, and N represents the total number of documents. TF-IDF can help find words with high discriminability for a specific document or context.

[0012] In a preferred embodiment, the specific steps for the video analysis technology in the data input processing module to extract video content are:

[0013] S1. Divide the video into a series of frames for processing each frame. Videos are usually dynamic, and frame extraction allows us to analyze the video content at each time point;

[0014] S2. Use a pre-trained CNN (such as ResNet, VGG, or EfficientNet) to extract features from each frame image. CNN can capture basic visual elements in the image, such as edges, textures, shapes, etc.;

[0015] S3. Use an object detection model (such as YOLO, Faster R-CNN, or RetinaNet) to identify specific objects in the video, including identifying human faces, vehicles, animals, or other objects;

[0016] S4. Use deep learning methods (such as Mask R-CNN, U-Net, etc.) to perform pixel-level image segmentation on video frames, extract the foreground and background in the scene for separation, and thus analyze the content of different regions.

[0017] In a preferred embodiment, the scoring dimensions of the automatic scoring calculation module include market feasibility, work completion, technical feasibility, team composition, psychological construction, cost investment, and industry influence.

[0018] In a preferred embodiment, the specific scoring process of the automatic scoring calculation module is as follows:

[0019] S1. Calculate the scores of each dimension based on the fused feature data and dimension standards transmitted by the data input processing module, and obtain the total score according to the weighted average. The calculation formula is:

[0020] S total = w1·S market + w2·S completion + w3·S tech + w4·S team + w5·S psych + w6·S cost + w7·S impact

[0021] where w1, w2, w3, w4, w5, w6, w7 respectively represent the weight coefficients of each dimension, S market represents market feasibility; S completion represents work completion; S tech represents technical feasibility; S team represents team composition; S psych represents psychological construction; S cost represents cost investment; S impact represents industry influence;

[0022] S2. Use regression analysis of the supervised learning algorithm to automatically adjust the dimension scores and weights according to historical data.

[0023] In a preferred embodiment, the automatic scoring results displayed in the expert interaction interface of the expert review simulation module include the scores of each dimension and their bases. Experts can view each step in the scoring process. At the same time, a radar chart is used to display the comparative analysis of the participating works in each dimension with the award-winning works in previous years to help experts make more accurate decisions.

[0024] In a preferred embodiment, the expert review simulation module continuously optimizes the review model through the reward function of reinforcement learning. The specific calculation formula of the reward function is:

[0025]

[0026] Among them, δ i represents the feedback weight, S i,feedback represents the expert feedback score, and A i,feedback represents the system score.

[0027] In a preferred embodiment, the auxiliary decision support module uses statistical analysis methods to deeply mine various data of the participating works, revealing potential trends and patterns. At the same time, through visual scoring reports, trend charts and industry analysis, it assists experts in making decisions.

[0028] In a preferred embodiment, the calculation formula of the multi-objective optimization algorithm is:

[0029]

[0030] Among them, x represents the candidate decision-making scheme, and ω i represents the weight of each dimension, and S i (x) represents the score of the decision-making scheme in the i-th dimension.

[0031] This application also provides a review method based on an adaptive scoring algorithm and multi-modal analysis, which specifically includes the following steps:

[0032] Step 1: Collect the text and video data of the participating works, use the NLP algorithm to extract text features, and use video analysis technology to extract visual features. Then, fuse the two features through the multi-modal BERT model to provide input for the scoring module;

[0033] Step 2: Based on the fused feature data and the preset scoring criteria, preliminarily score each participating work, and generate a scoring result as a reference for expert review;

[0034] Step 3: Build an expert scoring knowledge base. By simulating expert scoring and creating an interactive interface, experts can adjust the score and provide feedback to optimize the scoring criteria and improve the fairness of scoring;

[0035] Step 4: Assist experts in comprehensive analysis through data visualization and optimization tools to support experts in making accurate decisions.

[0036] The beneficial effects of the present invention are: it can dynamically adjust the scoring weight and criteria according to the specific requirements and standards of the competition. Through machine learning technology, it analyzes historical review data, identifies key review factors, and automatically adjusts the scoring model according to the characteristics and goals of the current competition. This adaptive ability enables the review system to more accurately evaluate the participating works and reduce human bias at the same time. Brief Description of the Drawings

[0037] Figure 1 This is the flowchart of the method of the present invention;

[0038] Figure 2 This is the block diagram of the system structure of the present invention. Detailed implementation manners

[0039] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0040] In the description of the present application, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of the described features. In the description of the present application, "a plurality" means two or more, unless otherwise specifically defined.

[0041] In the description of the present application, the term "for example" is used to mean "serving as an example, illustration, or explanation". Any embodiment described as "for example" in the present application is not necessarily construed as being more preferred or more advantageous than other embodiments. In order for any person skilled in the art to implement and use the present invention, the following description is given. In the following description, details are set forth for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid unnecessary details from obscuring the description of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed in the present application.

[0042] Embodiment 1

[0043] This embodiment provides a Figure 2 review system based on an adaptive scoring algorithm and multimodal analysis as shown in the figure, specifically including: a data input processing module, an automatic scoring calculation module, an expert review simulation module, and an auxiliary decision-making support module;

[0044] Data Input Processing Module: Obtain the text and video submission data of the College Students' Innovation and Entrepreneurship Competition, and use the TF-IDF of the natural language processing NLP algorithm to extract the key information in the text submission data. Use video analysis technology to extract the metadata and key visual information of the video content. Fuse the text feature vectors extracted by NLP and the visual feature vectors extracted by video analysis using the mid-term fusion method of the multi-modal BERT learning model. Use the shared Transformer structure to jointly represent the features of the text and video, enabling the model to capture the correlation between the two modalities, and transmit the fused features to the Automatic Scoring Calculation Module for scoring, providing the required input for subsequent scoring;

[0045] Automatic Scoring Calculation Module: After obtaining the fused feature data transmitted by the Data Input Processing Module, based on the preset scoring criteria, automatically perform a preliminary scoring result for each submission;

[0046] Expert Review Simulation Module: By organizing the basic principles and historical data of expert scoring, construct a knowledge base, and then generate simulated expert scores according to the information in the knowledge base. Create an expert interaction interface to display the automatic scoring results and allow experts to adjust and modify the scores. And all decision-making processes will be recorded, including the basis, data, and model parameters, ensuring the fairness and transparency of the scoring. As expert feedback is continuously incorporated, the scoring criteria are also continuously improved, ensuring the accuracy and fairness of the review results. At the same time, collect expert feedback, use reinforcement learning to adjust the scoring model, optimize the review process, and continuously optimize the scoring model and criteria according to expert feedback, continuously improving the quality and effect of the review process;

[0047] Auxiliary Decision Support Module: In the expert interaction interface, provide experts with data analysis and visualization tools, and use multi-objective optimization algorithms to help experts make optimized decisions according to the requirements of each scoring dimension, ensuring the comprehensiveness and accuracy of the review.

[0048] In this embodiment, specifically, it needs to be explained about the Data Input Processing Module. The TF-IDF calculation formula of the natural language processing NLP algorithm in the Data Input Processing Module is as follows:

[0049]

[0050] Where, t represents the word, d represents the document, tf(t, d) represents the term frequency of word t in document d, df(t) represents the number of documents containing word t, N represents the total number of documents, and TF-IDF can help find the words with high discrimination for a specific document or context;

[0051] The specific steps for the video analysis technology in the Data Input Processing Module to extract video content are as follows:

[0052] S1. Divide the video into a series of frames for processing each frame. Videos are usually dynamic, and frame extraction allows us to analyze the video content at each time point;

[0053] S2. Use a pre-trained CNN (such as ResNet, VGG, or EfficientNet) to extract features from each frame image. The CNN can capture basic visual elements in the image, such as edges, textures, shapes, etc.;

[0054] S3. Use an object detection model (such as YOLO, Faster R-CNN, or RetinaNet) to identify specific objects in the video, including identifying human faces, vehicles, animals, or other objects;

[0055] S4. Use deep learning methods (such as Mask R-CNN, U-Net, etc.) to perform pixel-level image segmentation on the video frames, extract the foreground and background separation in the scene, and thus analyze the content of different regions.

[0056] In this embodiment, specifically, it is necessary to explain the automatic scoring calculation module. The scoring dimensions of the automatic scoring calculation module include market feasibility, work completion, technical feasibility, team composition, psychological construction, cost investment, and industry influence;

[0057] The specific scoring process of the automatic scoring calculation module is as follows:

[0058] S1. Based on the fused feature data and dimension standards transmitted by the data input processing module, calculate the scores for each dimension and obtain the total score according to the weighted average. The calculation formula is:

[0059] S total = w1·S market + w2·S completion + w3·S tech + w4·S team + w5·S psych + w6·S cost + w7·S impact

[0060] Among them, w1, w2, w3, w4, w5, w6, w7 respectively represent the weight coefficients of each dimension, S market represents market feasibility; S completion represents work completion; S tech represents technical feasibility; S team represents team composition; S psych represents psychological construction; S cost represents cost investment; S impact represents industry influence;

[0061] S2. Regression analysis using a supervised learning algorithm automatically adjusts the dimension scores and weights based on historical data.

[0062] In this embodiment, specifically, it is necessary to explain the expert review simulation module. The automatic scoring results shown in the expert interaction interface of the expert review simulation module include the scores of each dimension and their basis. Experts can view each step in the scoring process. At the same time, the radar chart is used to display the comparative analysis of the participating works in each dimension with the winning works of previous years, helping experts make more accurate decisions.

[0063] The expert review simulation module continuously optimizes the review model through the reward function of reinforcement learning. The specific calculation formula of the reward function is:

[0064]

[0065] where δ i represents the feedback weight, S i,feedback represents the expert feedback score, and A i,feedback represents the system score.

[0066] In this embodiment, specifically, it is necessary to explain the auxiliary decision support module. The auxiliary decision support module uses statistical analysis methods to deeply mine the data of the participating works, revealing potential trends and patterns. At the same time, through visual scoring reports, trend charts and industry analysis, it assists experts in making decisions.

[0067] The calculation formula of the multi-objective optimization algorithm is:

[0068]

[0069] where x represents the candidate decision-making scheme, ω i represents the weights of each dimension, and S i (x) represents the score of the decision-making scheme in the i-th dimension.

[0070] Embodiment 2

[0071] This embodiment provides a review method based on an adaptive scoring algorithm and multi-modal analysis as shown in Figure 1 and specifically includes the following steps:

[0072] Step 1: Collect the text and video data of the participating works, use the NLP algorithm to extract text features, and use video analysis technology to extract visual features. Then, fuse the two features through the multi-modal BERT model to provide input for the scoring module.

[0073] Step 2: Based on the fused feature data and the preset scoring criteria, preliminarily score each participating work, and generate the scoring results as a reference for expert review.

[0074] Step 3: Build an expert scoring knowledge base. By simulating expert scoring and creating an interactive interface, experts can adjust the scores and provide feedback to optimize the scoring criteria and enhance the fairness of scoring.

[0075] Step 4: Assist experts in conducting comprehensive analysis through data visualization and optimization tools to support experts in making accurate decisions.

[0076] It should be noted that in the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0077] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0078] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded computers, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0079] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realizes the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide means for realizing the functions specified in Figure 1 one or more flows and / or blocksFigure 1 Steps of functions specified in one or more boxes.

[0081] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic inventive concept. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications that fall within the scope of the present invention.

[0082] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A review system based on an adaptive scoring algorithm and multimodal analysis, characterized in that Specifically, it includes: A data input processing module, an automatic scoring calculation module, an expert review simulation module, and an auxiliary decision-making support module; Data input processing module: Obtain the text and video entry data of the college students' innovation and entrepreneurship competition, and use the TF-IDF of the natural language processing NLP algorithm to extract the key information in the text entry data. Use video analysis technology to extract the metadata and key visual information of the video content. Combine the text feature vectors extracted by NLP and the visual feature vectors extracted by video analysis using the mid-term fusion method of the multi-modal BERT learning model. Use the shared Transformer structure to jointly represent the features of the text and video, and transmit the fused features to the automatic scoring calculation module for scoring; Automatic scoring calculation module: After obtaining the fused feature data transmitted by the data input processing module, based on the preset scoring criteria, automatically perform a preliminary scoring result for each entry; Expert review simulation module: By sorting out the basic principles and historical data of expert scoring, build a knowledge base, and then generate simulated expert scores according to the information in the knowledge base. Create an expert interaction interface to display the automatic scoring results and allow experts to adjust and modify the scores. And all decision-making processes will be recorded. At the same time, collect expert feedback and use reinforcement learning to adjust the scoring model to optimize the review process; Auxiliary decision-making support module: In the expert interaction interface, provide experts with data analysis and visualization tools, and use multi-objective optimization algorithms to help experts make optimized decisions according to the requirements of each scoring dimension.

2. The review system based on an adaptive scoring algorithm and multimodal analysis according to claim 1, wherein: The TF-IDF calculation formula of the natural language processing NLP algorithm in the data input processing module is: Where, t represents the word, d represents the document, tf(t, d) represents the word frequency of word t in document d, df(t) represents the number of documents containing word t, and N represents the total number of documents.

3. The review system based on an adaptive scoring algorithm and multimodal analysis according to claim 2, characterized in that: The specific steps for the video analysis technology in the data input processing module to extract video content are: S1. Divide the video into a series of frames for processing each frame; S2. Use a pre-trained CNN to extract features from each frame image; S3. Use an object detection model to identify specific objects in the video, including identifying human faces, vehicles, animals, or other objects; S4. Use deep learning methods to perform pixel-level image segmentation on the video frames to extract the foreground and background separation in the scene, thereby analyzing the content of different regions.

4. The review system based on an adaptive scoring algorithm and multimodal analysis according to claim 3, characterized in that: The scoring dimensions of the automatic scoring calculation module include market feasibility, work completion, technical feasibility, team composition, psychological construction, cost investment, and industry influence.

5. The review system based on an adaptive scoring algorithm and multimodal analysis according to claim 4, wherein: The specific scoring process of the automatic scoring calculation module is: S1. Based on the fused feature data and dimension standards transmitted by the data input processing module, calculate the scores of each dimension, and obtain the total score according to the weighted average. The calculation formula is: S total = w1·S market + w2·S completion + w3·S tech + w4·S team + w5·S psych + w6·S cost + w7·S impact Among them, w1, w2, w3, w4, w5, w6, w7 respectively represent the weight coefficients of each dimension, and S market represents market feasibility; S completion represents the degree of work completion; S tech represents technical feasibility; S team represents the team composition; S psych represents psychological construction; S cost represents cost investment; S impact represents industry influence; S2. Use regression analysis of the supervised learning algorithm to automatically adjust the dimension scores and weights according to historical data.

6. The review system based on an adaptive scoring algorithm and multimodal analysis according to claim 5, characterized in that: The automatic scoring results are displayed in the expert interaction interface of the expert review simulation module, including the scores for each dimension and their bases. Experts can view each step in the scoring process. At the same time, a radar chart is used to display the comparative analysis of the participating works in each dimension with the winning works of previous years to help experts make more accurate decisions.

7. An evaluation system based on an adaptive scoring algorithm and multimodal analysis according to claim 6, characterized in that: The expert review simulation module continuously optimizes the review model through the reward function of reinforcement learning. The specific calculation formula of the reward function is: Among them, δ i represents the feedback weight, S i,feedback represents the expert feedback score, and A i,feedback represents the system score.

8. The review system based on an adaptive scoring algorithm and multimodal analysis according to claim 7, characterized in that: The auxiliary decision-making support module uses statistical analysis methods to deeply mine the data of participating works, revealing potential trends and patterns. At the same time, through visual scoring reports, trend charts and industry analysis, it assists experts in making decisions.

9. The review system based on an adaptive scoring algorithm and multimodal analysis according to claim 8, wherein: The calculation formula of the multi-objective optimization algorithm is: Among them, x represents the candidate decision-making scheme, and ω i represents the weights of each dimension, and S i (x) represents the score of the decision-making scheme in the i-th dimension.

10. A review method based on an adaptive scoring algorithm and multimodal analysis is applied to a review system based on an adaptive scoring algorithm and multimodal analysis as described in any one of claims 1-9, characterized in that: Specifically, it includes the following steps: Step 1: Collect the text and video data of the participating works, use NLP algorithms to extract text features, and video analysis techniques to extract visual features. Then, the multi-modal BERT model is used to fuse the two features to provide input for the scoring module. Step 2: Based on the fused feature data and the preset scoring criteria, conduct a preliminary score for each participating work to generate a scoring result as a reference for expert review. Step 3: Build an expert scoring knowledge base. By simulating expert scoring and creating an interactive interface, experts can adjust the scores and provide feedback to optimize the scoring criteria and improve the fairness of scoring. Step 4: Through data visualization and optimization tools, assist experts in comprehensive analysis and support experts in making accurate decisions.

Images