Full-process review intelligent verification system and method based on multi-modal analysis

Abstract

The invention discloses an intelligent verification system and method for full-process review based on multi-modal analysis, and belongs to the technical field of intelligent review. Comprising a processing analysis module, an experiment extraction module, a risk early warning module, a reproduction verification module, a chart comparison module, a dispute tracing module, a feedback analysis module, a load monitoring module, a report generation module and a preference analysis module. According to the invention, comprehensive and multi-angle intelligent review can be realized, more detailed risk early warning and positioning information can be generated, the experiment reproducibility can be quickly verified at low cost, the manual verification burden is greatly reduced, the innovation and risk of the thesis viewpoint are scientifically evaluated, the fatigue of a manuscript reviewer is relieved, the manuscript review efficiency and quality are improved, and the method is suitable for popularization and application. And dynamically adjusting the review content recommendation sequence.

Description

Technical Field

[0001] The present invention relates to the field of intelligent review technology, and in particular to a full-process review intelligent verification system and method based on multimodal analysis. Background Art

[0002] With the continued expansion of global scholarly publishing and the rise of interdisciplinary and cross-disciplinary research, the complexity and challenges facing academic review have significantly increased. On the one hand, the manifestations of academic misconduct have become increasingly diverse, evolving from early instances of plagiarism and duplicate publication to more subtle and complex behaviors such as data manipulation, figure fabrication, algorithmic plagiarism, irreproducible results, and concealment of conflicting interests. On the other hand, the presentation of current academic papers is becoming increasingly multimodal. In addition to traditional textual narratives, a large number of research results include experimental data, mathematical formulas, executable code, visualizations, pseudocode frameworks, and interactive components. Traditional verification tools that primarily rely on text comparison are no longer able to cope with this complex review process. Furthermore, the issue of experimental reproducibility is becoming a focus of global scientific research. Numerous studies have shown that across multiple fields, the proportion of experimental results that cannot be independently replicated remains high, resulting in a waste of academic resources and hindering the normal advancement of scientific progress. Furthermore, excessive cognitive load on reviewers and low review efficiency are long-standing bottlenecks in the academic publishing process. Faced with a massive amount of papers and highly complex experimental details, it is difficult for reviewers to comprehensively evaluate the quality of the papers within a limited time, resulting in uneven review quality, frequent review bias, and missed reviews.

[0003] The existing full-process review intelligent verification system and method cannot achieve comprehensive and multi-angle intelligent review. The time and cost of verifying the reproducibility of the experiment are long, which greatly increases the burden of manual verification. Reviewer fatigue cannot be effectively alleviated, which reduces the efficiency and quality of review. At the same time, the order of recommended review content is dynamically adjusted. To this end, we propose a full-process review intelligent verification system and method based on multimodal analysis. Summary of the Invention

[0004] The purpose of the present invention is to solve the defects existing in the prior art and to propose a full-process review intelligent verification system and method based on multimodal analysis.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions: A full-process review and intelligent verification system based on multimodal analysis, including a processing and analysis module, an experiment extraction module, a risk warning module, a reproduction and verification module, a chart comparison module, a dispute tracing module, a feedback analysis module, a load monitoring module, a report generation module, and a preference analysis module; The processing and parsing module is used to parse the multimodal content of the paper text, charts, pseudocode, formulas, and references to extract structural elements; The experimental extraction module generates a standardized experimental protocol tree based on the method description, experimental steps, mathematical formulas and equipment configuration information in the paper; The risk warning module analyzes the experimental data, charts and pseudocode in the paper, and parses the references to predict potential risk information of the paper; The reproducibility verification module generates an executable code container based on the experimental protocol tree to run the experiment, compares the differences between the paper results and the actual execution results, and outputs a reproducibility analysis report; The chart comparison module is used to extract chart data, curve trends, and statistical indicators in the paper, and cross-check them with the text description and experimental data; The controversy tracing module extracts conclusions and potentially controversial viewpoints in papers and evaluates the innovation and risk of the conclusions through historical documents and academic social networks. The feedback analysis module is used to collect feedback on the target paper, identify valid doubts and invalid criticisms, and generate a dispute credibility score; The load monitoring module is used to analyze the cognitive load level of reviewers during the review process in real time and dynamically adjust the review mode; The report generation module is used to integrate test results, verification reports, risk warnings, dispute analysis, and cognitive load monitoring data, and generate a corresponding review comprehensive analysis report; The preference analysis module is used to analyze the reviewer's review tendency and potential bias based on the reviewer's past review records, decision-making preferences, and cognitive load characteristics.

[0006] As a further solution of the present invention, the specific steps of extracting structural elements by the processing and parsing module are as follows: S1.1: Convert the paper PDF document into a parseable XML, JSON, or TXT format. Use the text block segmentation scoring function to segment the converted paper content into paragraphs, remove headers and footers, extract font information, and perform layout analysis. S1.2: Use natural language processing technology to identify the chapter structure of the paper, then extract the basic information of each chapter structure, including title, author, institution, abstract, keywords, and chapter title. Then detect the image areas in the paper and distinguish between charts, formula diagrams, and illustrations. S1.3: Vectorize and extract the coordinate axes, curves, and bar chart elements in the graphs and reconstruct the corresponding digital data. Then, locate the pseudo-code blocks in the paper based on the typesetting characteristics. Then, calculate the confidence score of the located pseudo-code blocks. If the confidence score is higher than the preset threshold, the block is judged to be a code block. Otherwise, feedback is provided to the reviewer for verification. S1.4: Use syntax tree parsing technology to convert each pseudocode into a structured syntax tree representation. Then, identify mathematical formula images or LaTeX code and convert them into MathML structured formula representations. Extract variables, operators, and mathematical symbols from the formulas and construct a symbol relationship diagram. S1.5: Extract the author, publication year, article title, journal name, and page number information from the reference block, and parse the extracted information in the reference block based on standardized rules.

[0007] As a further solution of the present invention, the specific steps of the risk warning module for predicting potential risk information of a paper are as follows: S2.1: Extract experimental data from the paper, including numerical values ​​in tables and graphs, and data descriptions in the main text. Analyze data distribution characteristics using statistical anomaly detection techniques to identify abnormal deviations, abnormal concentrations, and regular repetitions. Based on the analysis results, warn of potential data falsification in the current paper. S2.2: Calculate the numerical similarity of all numbers, data paragraphs, and chart data that appear in the paper. If the numerical similarity is higher than a preset threshold, it is judged that the experimental data is reused. At the same time, the numerical reuse rate of the entire paper is calculated. If the numerical reuse rate is higher than a preset value, it is judged that the paper has a risk of data misuse. S2.3: Analyze the statistical test results in the paper and calculate the consistency error between each statistical test result and the data. If the consistency error is higher than the preset threshold, it is judged that there are false statistics or fabricated results in the paper, and the corresponding statistical test results paragraph in the paper will be marked; S2.4: Calculate the image overlap detection index for each image, chart, and experimental photograph extracted from the paper. Based on the image overlap detection index, analyze the pixel-level consistency and noise pattern changes. If the image overlap detection index is higher than a preset threshold, the corresponding image, chart, or experimental photograph is judged to have traces of image tampering and marked. S2.5: Extract author information from the papers and their citations from the reference block, and use the extracted authors of each citation as a node. Then, construct an author relationship network based on the number of co-publications and co-citations between authors, and analyze the historical frequency of collaboration, number of co-publications, and cross-citation relationships between authors. S2.6: Extract the institutional information, funding information, and collaborative institution records of the authors of each cited article. Construct an institutional interest relationship matrix based on the number of shared funding projects, the number of co-authors, and the total number of papers collaborated between the two institutions. Based on this institutional interest relationship matrix, analyze the potential overlap of interests between institutions. S2.7: Based on the author relationship network and the institutional interest relationship matrix, calculate the probability of conflict of interest between authors or institutions, and establish a corresponding conflict of interest probability matrix. At the same time, output the author pairs, institution pairs, and author-institution pairs with conflict of interest probabilities higher than the preset values ​​in tabular form, and generate corresponding warning signals.

[0008] As a further solution of the present invention, the specific construction formula of the author relationship network described in S2.5 is as follows: Where, Representative Authors With the author the intensity of collaboration between them; Representative Authors With the author Number of co-published papers; Representative Authors With the author the number of co-citations; Representing the authors With the author Total number of published papers respectively; The specific formula for constructing the institutional interest relationship matrix described in S2.6 is as follows: Where, The strength of interest linkages among representatives of institutions; The number of mutual fund projects among representative institutions; The number of co-authors between representative institutions; The total number of papers representing collaboration between the two institutions; Represents a small constant that prevents division by zero errors; The specific calculation formula for the probability of conflict of interest described in S2.7 is as follows: Where, the probability of conflict of interest between the representative authors or institutions; Represents the adjustment coefficient.

[0009] 5. The full-process review intelligent verification system based on multimodal analysis according to claim 3 is characterized in that the specific steps of the load monitoring module for real-time analysis of the cognitive load level of the reviewer during the review process are as follows: S3.1: During the review process, the load monitoring module simultaneously records the reviewer's eye movement data, operation data, and voice review record behavior data, and standardizes the collected behavior data. Then, based on the processed behavior data, the comprehensive cognitive load level is calculated. The specific calculation formula for the comprehensive cognitive load level is as follows: Where, Represents the current comprehensive score of cognitive load; represents the comprehensive score of eye movement indicators; represents the operation complexity score; represents the speech load feature score; Represents the current review duration; Represent weight coefficients respectively; S3.2: If If the cognitive load exceeds the preset threshold, the reviewer's cognitive load is judged to be too high, and the fragmented review mode is triggered. At the same time, the paper automatically selects the paragraphs, potentially controversial viewpoints, charts, and core formulas that need to be reviewed first, and recommends a review order for the reviewer based on the content priority from high to low; S3.2: Continuously track the reviewers’ feedback behaviors, including the length of time they stay in the fragmented review mode, whether they mark the manuscript as “reviewed”, and whether they propose revisions, and dynamically adjust the priority of recommended content based on each feedback behavior.

[0010] The full-process review and intelligent verification method based on multimodal analysis has the following specific steps: Ⅰ. Analyze various information in the paper and extract multimodal information in the paper. Based on the extracted multimodal information, build a corresponding paper analysis database; II. Identify and extract the experimental process in the paper, establish a standardized experimental protocol tree, and identify the academic ethical risks in the paper; III. After completing the method extraction and ethics review, create the corresponding experimental environment based on the generated experimental protocol tree, replicate the paper experiment, and evaluate the credibility of the experiment; IV. After the experimental replication is completed, the historical evolution path and controversial records of the paper’s viewpoints are tracked, and a spatiotemporal map of academic controversy is generated to analyze the controversial risks of the paper’s conclusions; V. Integrate real-time feedback from different sources, identify different types of feedback, distinguish rational criticism from emotional speech, and generate a credibility score for the paper's controversy; VI. Summarize all data from the verification process and generate a comprehensive verification report for the corresponding paper, while providing visual charts, dispute tracing path diagrams and experimental comparison results.

[0011] As a further solution of the present invention, the specific steps for reproducing the experiment in step III are as follows: S4.1: Parse the experimental method descriptions, mathematical formulas, and experimental configuration sections in the paper and extract the experimental operation steps, variable settings, model parameters, and hardware environment information. Calculate the dependency weights of each experimental operation step and organize them into a standardized experimental protocol tree based on the experimental process, variable dependencies, and control flow. S4.2: Expand each experimental step node from the constructed standardized experimental protocol tree, extract the complete execution path, and determine the operation sequence, input and output dependencies, and execution conditions of each node. Generate a corresponding operation sequence list. Then, map the input and output variables, control parameters, and hyperparameters in the standardized experimental protocol tree to variable names in the selected programming language, and establish binding relationships between variables and experimental operations. S4.3: Translate the algorithm descriptions, mathematical formulas, and pseudocode in the protocol tree into executable function modules in the program. During the translation process, the semantic consistency of each algorithm logic translation is calculated in real time. If the consistency falls below the preset threshold, it indicates a translation anomaly and the reviewer is requested to make manual adjustments. The corresponding function definition, loop, conditional judgment, and data processing flow are then generated according to the experimental logic. S4.4: Assemble the generated function modules, variable binding sections, pre-processing and post-processing processes into a complete executable script in the order of the execution path, and insert the required intermediate variable initialization, error handling statements, and log output modules to generate a complete experimental code; S4.5: Based on the programming language, operating system, and dependency framework involved in the experimental code, select the corresponding basic Docker container and initialize the container configuration. After initialization, parse the library dependency declarations in the experimental code and the dependency list extracted from the protocol tree, and install the corresponding dependency packages according to the dependency list. Then, automatically mount the experimental code, scripts, configuration files, and test data into the specified directory in the Docker container; S4.6: Automatically execute the generated experimental container in the Docker container, run the complete experimental process, extract the experimental result data, logs and output files generated in real time, and automatically compare the experimental data reported in the paper with the actual experimental results in the Docker container; S4.7: Based on the comparison results, evaluate the average difference between the actual experimental results and the experimental data reported in the paper, summarize the protocol tree structure, generated code, container configuration, run logs, and experimental comparison results, and generate a standardized reproducibility analysis report.

[0012] As a further solution of the present invention, the specific steps of analyzing the controversial risk of the paper conclusions in step IV are as follows: S5.1: Evaluate the subversiveness score of each conclusion paragraph, discussion paragraph, and abstract in the paper, and select sentences or paragraphs with subversiveness scores above a preset threshold. Then, search the historical paper database for historical viewpoints that correspond to or oppose the current viewpoint, and calculate the strength of the correlation between the current paper's viewpoint and the historical viewpoints. S5.2: Use historical viewpoints and current paper viewpoints as edges to build a semantic association network between papers. Annotate the inheritance, citation, or opposition relationships between viewpoints. Represent the relationships between papers as edges, using the strength of the association between each viewpoint as the weight of the corresponding edge. Collect discussions about the current paper viewpoint on different academic social platforms. S5.3: Extract the various viewpoints in the discussion content, including questions, defenses, and supplementary information, and mark the discussion time and user influence of each viewpoint. Calculate the social controversy heat of the current paper's viewpoint. Based on the viewpoint evolution data in historical documents and the academic social discussion timeline, calculate the propagation path span of the current paper's viewpoint, and generate the spatiotemporal evolution trajectory of the viewpoint. S5.4: Based on the generated spatiotemporal evolution trajectory, construct a time series diagram and a cross-institutional and cross-disciplinary communication path diagram. Use the publication time, geographic location, opinion dissemination channel, and the citation, inheritance, and opposition relationships between opinions as node data in the communication path diagram, and calculate the spatiotemporal density weight of each node data. S5.5: Based on the spatiotemporal density weights of each node data, distribute the nodes on the time axis and geographic coordinate system, and use different colors and shapes to mark the opinion type of each node. Then, use connecting lines to represent the association paths between opinions to generate a dispute map for the corresponding opinion. Calculate the dispute peak value for each area in each dispute map, and mark the area with a dispute peak value above a preset threshold as a dispute peak area; S5.6: Analyze the propagation paths of controversial viewpoints in each dispute peak area from their origin to the current state, and evaluate the complexity of the propagation paths. Based on the sum of the spatiotemporal density weights, the dispute peak, the complexity of the propagation paths, and the subversiveness of the initial viewpoints, calculate the dispute risk score of the current paper's viewpoints, and output a dispute risk assessment report.

[0013] Compared with the prior art, the present invention has the following beneficial effects: The present invention extracts structured data from paper text, charts, images, pseudocode, formulas, and references, and detects academic misconduct risks such as data tampering, reuse, falsified statistics, and image tampering, generating an ethical warning report. Simultaneously, it extracts experimental methods, variable settings, and model parameters, constructs a standardized experimental protocol tree, and automatically generates an executable code container based on the constructed standardized experimental protocol. The experiment is run in an isolated environment, automatically comparing the paper with the actual results, and outputting a reproducibility analysis report. The present invention then tracks disruptive viewpoints in the paper, combines historical documents with academic social data, constructs a viewpoint dispute map, plots the spatiotemporal propagation trajectory, and assesses the risk of dispute. Furthermore, during the review process, the system monitors the reviewer's eye movements, operations, and voice data in real time, calculates cognitive load, and intelligently adjusts the review process. This enables comprehensive, multi-angle intelligent review, generates more detailed risk warnings and location information, quickly and cost-effectively verifies experimental reproducibility, significantly reduces the burden of manual verification, scientifically evaluates the innovation and risk of the paper's viewpoints, alleviates reviewer fatigue, improves review efficiency and quality, and dynamically adjusts the order of recommended review content. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0015] Figure 1 This is a system block diagram of the full-process review and intelligent verification system based on multimodal analysis proposed by the present invention; Figure 2 This is a flowchart of the full-process review and intelligent verification method based on multimodal analysis proposed in the present invention. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0017] Example 1 Reference Figure 1 , a full-process review intelligent verification system based on multimodal analysis, including processing and analysis module, experimental extraction module, risk warning module, reproduction verification module, chart comparison module, dispute tracing module, feedback analysis module, load monitoring module, report generation module and preference analysis module.

[0018] The processing and parsing module is used to parse the multimodal content of paper text, charts, pseudocode, formulas, and references, and extract structured elements.

[0019] Specifically, the paper PDF document is converted into a parsable XML, JSON or TXT format, and the text block division scoring function is used to segment the converted paper content into paragraphs, remove headers and footers, extract font information and analyze the layout. Through natural language processing technology, the chapter structure in the paper is identified, and then the title, author, institution, abstract, keywords and chapter title of each chapter structure are extracted. Then, the image area in the paper is detected, and the charts, formula charts and illustrations are distinguished. The coordinate axes, curves and bar charts in the charts are vectorized and the corresponding digital data are reconstructed. Then, the paper is located according to the layout characteristics. The pseudocode blocks in the text are then analyzed, and the confidence score of the located pseudocode blocks is calculated. If the confidence score is higher than the preset threshold, the block is judged to be a code block. Otherwise, it is fed back to the reviewer for verification. Through syntax tree parsing technology, each pseudocode is converted into a structured syntax tree representation, and then the mathematical formula image or LaTeX code is identified and converted into a MathML structured formula representation. At the same time, the variables, operators and mathematical symbols in the formula are extracted, and a symbol relationship diagram is constructed. The author, publication year, article title, journal name and page number information in the reference block are extracted, and the information in the extracted reference block is parsed based on standardized rules.

[0020] The experiment extraction module generates a standardized experimental protocol tree based on the method description, experimental steps, mathematical formulas and equipment configuration information in the paper; the risk warning module analyzes the experimental data, charts and pseudocode in the paper, and parses the references to predict potential risk information in the paper.

[0021] Specifically, extract the experimental data in the paper, including the numerical values ​​in the tables and charts and the data descriptions in the text, and then use statistical anomaly detection technology to analyze the data distribution characteristics, identify abnormal deviations, abnormal concentrations and regular repetitions, and warn of potential data falsification in the current paper based on the analysis results. Calculate the numerical similarity of all numbers, data paragraphs, and chart data that appear in the paper. If the numerical similarity is higher than the preset threshold, it is judged that the experimental data is reused. At the same time, calculate the numerical reuse rate of the full text of the paper. If the numerical reuse rate is higher than the preset value, it is judged that the paper has a risk of data abuse. Analyze the statistical test results in the paper, and calculate the consistency error between each statistical test result and the data. If the consistency error is higher than the preset threshold, it is judged that there are false statistics or falsified results in the paper, and mark the corresponding statistical test result paragraphs of the paper. Calculate the image overlap detection index of each image, chart and experimental photo extracted from the paper. According to the image overlap detection index, analyze the pixel level consistency and noise pattern changes. If there is a high image overlap detection index If the threshold is lower than the preset threshold, the corresponding image, chart or experimental photo is judged to have traces of image tampering, and the corresponding image, chart or experimental photo is marked. The author information of the paper and its cited documents is extracted from the reference block, and the authors of each extracted cited document are used as nodes. Then, according to the number of co-published papers and the number of co-cited documents between the authors, the author relationship network is constructed, and the historical cooperation frequency, co-publication number and cross-reference relationship between the authors are analyzed. The information of the institutions to which the authors of each cited document belong, the funding information and the records of the cooperative institutions are extracted. At the same time, according to the number of shared fund projects, the number of co-authors and the total number of papers cooperated by the two institutions, the institutional interest relationship matrix is ​​constructed. At the same time, based on the institutional interest relationship matrix, the potential interest intersections between institutions are analyzed. According to the author relationship network and the institutional interest relationship matrix, the probability of conflict of interest between authors or institutions is calculated, and a corresponding conflict of interest probability matrix is ​​established. At the same time, the author pairs, institution pairs and author-institution pairs with conflict of interest probabilities higher than the preset values ​​are output in tabular form, and corresponding warning signals are generated.

[0022] It should be further explained that the specific construction formula of the author relationship network is as follows: Where, Representative Authors With the author the intensity of collaboration between them; Representative Authors With the author Number of co-published papers; Representative Authors With the author the number of co-citations; and Representing the authors With the author Total number of published papers respectively; The specific construction formula of the institutional interest relationship matrix is ​​as follows: Where, The strength of interest linkages among representatives of institutions; The number of mutual fund projects among representative institutions; The number of co-authors between representative institutions; The total number of papers representing collaboration between the two institutions; Represents a small constant that prevents division by zero errors; The specific calculation formula for the probability of conflict of interest is as follows: Where, the probability of conflict of interest between the representative authors or institutions; Represents the adjustment coefficient.

[0023] The reproducibility verification module generates an executable code container based on the experimental protocol tree to run the experiment, compares the differences between the paper results and the actual execution results, and outputs a reproducibility analysis report; The chart comparison module is used to extract chart data, curve trends, and statistical indicators in the paper, and cross-compare them with the text description and experimental data; the dispute tracing module extracts the conclusions and potentially controversial views in the paper, and evaluates the innovation and risk of the conclusions through historical documents and academic social networks.

[0024] The feedback analysis module is used to collect feedback on the target paper, identify valid questions and invalid criticisms, and generate a dispute credibility score; the load monitoring module is used to analyze the reviewer's cognitive load level during the review process in real time and dynamically adjust the review mode.

[0025] Specifically, during the review process, the load monitoring module synchronously records the reviewer's eye movement data, operation data, and voice review record behavioral data, and standardizes the collected behavioral data. Then, based on the processed behavioral data, the comprehensive cognitive load level is calculated. If the comprehensive cognitive load level is higher than the preset cognitive load threshold, the reviewer's cognitive load is judged to be too high, and the fragmented review mode is triggered. At the same time, the reviewer's paragraphs, potentially controversial viewpoints, charts, and core formulas that need to be reviewed first are automatically screened from the paper, and the review order is recommended to the reviewer based on the content priority from high to low. The module continuously tracks the reviewer's stay time in the fragmented review mode, whether it is marked as "reviewed", and whether it proposes modification opinions, and dynamically adjusts the priority of recommended content based on each feedback behavior.

[0026] As a further solution of this embodiment, the specific calculation formula of the comprehensive cognitive load level is as follows: Where, Represents the current comprehensive score of cognitive load; represents the comprehensive score of eye movement indicators; represents the operation complexity score; represents the speech load feature score; Represents the current review duration; Represent the weight coefficients respectively.

[0027] The report generation module is used to integrate test results, verification reports, risk warnings, dispute analysis, and cognitive load monitoring data, and generate corresponding review comprehensive analysis reports; the preference analysis module is used to analyze the reviewer's review tendencies and potential biases based on his or her past review records, decision-making preferences, and cognitive load characteristics.

[0028] Example 2 Reference Figure 2 , a full-process review intelligent verification method based on multimodal analysis, the specific steps of this verification method are as follows: Analyze various information in the paper and extract multimodal information in the paper. Based on the extracted multimodal information, build the corresponding paper analysis database.

[0029] Identify and extract the experimental process in the paper, establish a standardized experimental protocol tree, and identify the various academic ethical risks in the paper.

[0030] After completing the method extraction and ethical verification, the corresponding experimental environment is created based on the generated experimental protocol tree, the paper experiment is reproduced, and the credibility of the experiment is evaluated.

[0031] Specifically, parse the experimental method description, mathematical formulas and experimental configuration paragraphs in the paper, and extract the experimental operation steps, variable settings, model parameters and hardware environment information, then calculate the dependency weight of each experimental operation step, and then organize it into a standardized experimental protocol tree based on the experimental process, variable dependency and control process. From the constructed standardized experimental protocol tree, expand each experimental step node, extract the complete execution path, and determine the operation sequence, input and output dependency and execution conditions of each node, generate the corresponding operation sequence list, and then map the input and output variables, control parameters and hyperparameters in the standardized experimental protocol tree to the variable names in the selected programming language, and establish the binding relationship between the variables and the experimental operations. Translate the algorithm description, mathematical formulas and pseudocode in the protocol tree into executable function modules in the program. During the translation process, calculate the semantic consistency of the logical translation of each algorithm in real time. If the consistency is lower than the preset threshold, it indicates that the translation is abnormal, and the reviewer is requested to make manual adjustments. Then generate the corresponding function definition, loop, conditional judgment, and data processing process according to the experimental logic. The binding segment, preprocessing and post-processing processes are spliced ​​into a complete executable script in the order of the execution path, and the required intermediate variable initialization, error handling statements and log output modules are inserted to generate a complete experimental code. According to the programming language, operating system and dependency framework involved in the experimental code, the corresponding basic Docker container is selected and the container configuration is initialized. After the initialization is completed, the library dependency declaration in the experimental code and the dependency list extracted from the protocol tree are parsed, and the corresponding dependency package is installed according to the dependency list. Then, the experimental code, scripts, configuration files and test data are automatically mounted to the specified directory in the Docker container. The generated experimental container is automatically executed in the Docker container, the complete experimental process is run, the real-time generated experimental result data, logs and output files are extracted, and the experimental data of the paper report are automatically compared with the actual experimental results in the Docker container. Based on the comparison results, the average difference between the actual experimental results and the experimental data reported in the paper is evaluated. The protocol tree structure, generated code, container configuration, operation logs and experimental comparison results are summarized to generate a standardized reproducibility analysis report.

[0032] After the experiment is replicated, the historical evolution of the paper's viewpoints and controversial records are tracked, and a spatiotemporal map of academic disputes is generated to analyze the controversial risks of the paper's conclusions.

[0033] Specifically, evaluate the subversive score of each conclusion paragraph, discussion paragraph and abstract in the paper, and screen out sentences or paragraphs with subversive score higher than the preset threshold. Then, retrieve historical viewpoints corresponding to or opposite to the current viewpoint in the historical paper database, and calculate the viewpoint correlation strength between the current paper viewpoint and the historical viewpoint. Use historical viewpoints and current paper viewpoints as edges to establish a semantic association network between papers, mark the inheritance, citation or opposition relationship between viewpoints, and represent the relationship between papers with edges. Use the correlation strength of each viewpoint as the weight value of the corresponding edge, collect discussion content about the current paper viewpoint on different academic social platforms, extract various viewpoints in the discussion content, including questioning, defense and supplementary information, and mark the discussion time and user influence of each multiple viewpoint. At the same time, calculate the social controversy heat of the current paper viewpoint, and calculate the propagation path span of the current paper viewpoint based on the viewpoint evolution data in historical documents and the academic social discussion timeline, and generate the time and space of the viewpoint. Evolution trajectory, based on the generated spatiotemporal evolution trajectory, construct a time series diagram and a cross-institutional and cross-disciplinary communication path diagram, and use the publication time, geographical location, opinion dissemination channel, and the citation, inheritance, and opposition relationships between opinions as node data in the communication path diagram, and at the same time calculate the spatiotemporal density weight of each node data, and according to the spatiotemporal density weight of each node data, distribute points in the time axis and geographic coordinate system, and use different colors and shapes to mark the opinion type of each node, and then use connecting lines to represent the association path between opinions to generate a dispute map of the corresponding opinions, calculate the dispute peak value of each area in each dispute map, and mark the area where the dispute peak value is higher than the preset threshold as the dispute peak area, analyze the controversial opinion in each dispute peak area from the origin to the current propagation path, and evaluate the complexity of the propagation path, and calculate the dispute risk score of the current paper opinion based on the sum of the spatiotemporal density weights, the dispute peak value, the complexity of the propagation path and the subversiveness of the initial opinion, and output a dispute risk assessment report.

[0034] Integrate real-time feedback from different sources, identify different types of feedback information, distinguish rational criticism from emotional speech, and then generate a controversy credibility score for the paper.

[0035] Summarize all data from the verification process and generate a comprehensive verification report for the corresponding paper, while providing visual charts, dispute tracing path diagrams and experimental comparison results.

Claims

1. A full-process review and intelligent verification system based on multimodal analysis, characterized by: It includes processing and analysis module, experiment extraction module, risk warning module, reproduction verification module, chart comparison module, dispute tracing module, feedback analysis module, load monitoring module, report generation module and preference analysis module; The processing and parsing module is used to parse the multimodal content of the paper text, charts, pseudocode, formulas, and references to extract structural elements; The experimental extraction module generates a standardized experimental protocol tree based on the method description, experimental steps, mathematical formulas and equipment configuration information in the paper; The risk warning module analyzes the experimental data, charts and pseudocode in the paper, and parses the references to predict potential risk information of the paper; The reproducibility verification module generates an executable code container based on the experimental protocol tree to run the experiment, compares the differences between the paper results and the actual execution results, and outputs a reproducibility analysis report; The chart comparison module is used to extract chart data, curve trends, and statistical indicators in the paper, and cross-check them with the text description and experimental data; The controversy tracing module extracts conclusions and potentially controversial viewpoints in papers and evaluates the innovation and risk of the conclusions through historical documents and academic social networks. The feedback analysis module is used to collect feedback on the target paper, identify valid doubts and invalid criticisms, and generate a dispute credibility score; The load monitoring module is used to analyze the cognitive load level of reviewers during the review process in real time and dynamically adjust the review mode; The report generation module is used to integrate test results, verification reports, risk warnings, dispute analysis, and cognitive load monitoring data, and generate a corresponding review comprehensive analysis report; The preference analysis module is used to analyze the reviewer's review tendency and potential bias based on the reviewer's past review records, decision-making preferences, and cognitive load characteristics.

2. The full-process review and intelligent verification system based on multimodal analysis according to claim 1 is characterized in that: The specific steps of extracting structural elements by the processing and parsing module are as follows: S1.1: Convert the paper PDF document into a parseable XML, JSON, or TXT format. Use the text block segmentation scoring function to segment the converted paper content into paragraphs, remove headers and footers, extract font information, and perform layout analysis. S1.2: Use natural language processing technology to identify the chapter structure of the paper, then extract the basic information of each chapter structure, including title, author, institution, abstract, keywords, and chapter title. Then detect the image areas in the paper and distinguish between charts, formula diagrams, and illustrations. S1.3: Vectorize and extract the coordinate axes, curves, and bar chart elements in the graphs and reconstruct the corresponding digital data. Then, locate the pseudo-code blocks in the paper based on the typesetting characteristics. Then, calculate the confidence score of the located pseudo-code blocks. If the confidence score is higher than the preset threshold, the block is judged to be a code block. Otherwise, feedback is provided to the reviewer for verification. S1.4: Use syntax tree parsing technology to convert each pseudocode into a structured syntax tree representation. Then, identify mathematical formula images or LaTeX code and convert them into MathML structured formula representations. Extract variables, operators, and mathematical symbols from the formulas and construct a symbol relationship diagram. S1.5: Extract the author, publication year, article title, journal name, and page number information from the reference block, and parse the extracted information in the reference block based on standardized rules.

3. The full-process review and intelligent verification system based on multimodal analysis according to claim 2 is characterized in that: The specific steps of the risk warning module for predicting potential risk information of papers are as follows: S2.1: Extract experimental data from the paper, including numerical values ​​in tables and graphs, and data descriptions in the main text. Analyze data distribution characteristics using statistical anomaly detection techniques to identify abnormal deviations, abnormal concentrations, and regular repetitions. Based on the analysis results, warn of potential data falsification in the current paper. S2.2: Calculate the numerical similarity of all numbers, data paragraphs, and chart data that appear in the paper. If the numerical similarity is higher than a preset threshold, it is judged that the experimental data is reused. At the same time, the numerical reuse rate of the entire paper is calculated. If the numerical reuse rate is higher than a preset value, it is judged that the paper has a risk of data misuse. S2.3: Analyze the statistical test results in the paper and calculate the consistency error between each statistical test result and the data. If the consistency error is higher than the preset threshold, it is judged that there are false statistics or fabricated results in the paper, and the corresponding statistical test results paragraph in the paper will be marked; S2.4: Calculate the image overlap detection index for each image, chart, and experimental photograph extracted from the paper. Based on the image overlap detection index, analyze the pixel-level consistency and noise pattern changes. If the image overlap detection index is higher than a preset threshold, the corresponding image, chart, or experimental photograph is judged to have traces of image tampering and marked. S2.5: Extract author information from the papers and their citations from the reference block, and use the extracted authors of each citation as a node. Then, construct an author relationship network based on the number of co-publications and co-citations between authors, and analyze the historical frequency of collaboration, number of co-publications, and cross-citation relationships between authors. S2.6: Extract the institutional information, funding information, and collaborative institution records of the authors of each cited article. Construct an institutional interest relationship matrix based on the number of shared funding projects, the number of co-authors, and the total number of papers collaborated between the two institutions. Based on this institutional interest relationship matrix, analyze the potential overlap of interests between institutions. S2.7: Based on the author relationship network and the institutional interest relationship matrix, calculate the probability of conflict of interest between authors or institutions, and establish a corresponding conflict of interest probability matrix. At the same time, output the author pairs, institution pairs, and author-institution pairs with conflict of interest probabilities higher than the preset values ​​in tabular form, and generate corresponding warning signals.

4. The full-process review and intelligent verification system based on multimodal analysis according to claim 3 is characterized in that: The specific formula for constructing the author relationship network described in S2.5 is as follows: Where, Representative Authors With the author the intensity of collaboration between them; Representative Authors With the author Number of co-published papers; Representative Authors With the author the number of co-citations; Representing the authors With the author Total number of published papers respectively; The specific formula for constructing the institutional interest relationship matrix described in S2.6 is as follows: Where, The strength of interest linkages among representatives of institutions; The number of mutual fund projects among representative institutions; The number of co-authors between representative institutions; The total number of papers representing collaboration between the two institutions; Represents a small constant that prevents division by zero errors; The specific calculation formula for the probability of conflict of interest described in S2.7 is as follows: Where, the probability of conflict of interest between the representative authors or institutions; Represents the adjustment coefficient.

5. The full-process review and intelligent verification system based on multimodal analysis according to claim 3 is characterized in that: The specific steps of the load monitoring module for real-time analysis of the cognitive load level of reviewers during the review process are as follows: S3.1: During the review process, the load monitoring module simultaneously records the reviewer's eye movement data, operation data, and voice review record behavior data, and standardizes the collected behavior data. Then, based on the processed behavior data, the comprehensive cognitive load level is calculated. The specific calculation formula for the comprehensive cognitive load level is as follows: Where, Represents the current comprehensive score of cognitive load; represents the comprehensive score of eye movement indicators; represents the operation complexity score; represents the speech load feature score; Represents the current review duration; Represent weight coefficients respectively; S3.2: If If the cognitive load exceeds the preset threshold, the reviewer's cognitive load is judged to be too high, and the fragmented review mode is triggered. At the same time, the paper automatically selects the paragraphs, potentially controversial viewpoints, charts, and core formulas that need to be reviewed first, and recommends a review order for the reviewer based on the content priority from high to low; S3.2: Continuously track the reviewers' feedback behaviors, including the length of time they stay in the fragmented review mode, whether they mark "reviewed", and whether they propose revisions, and dynamically adjust the priority of recommended content based on each feedback behavior.

6. A full-process review intelligent verification method based on multimodal analysis, used to implement the full-process review intelligent verification system function based on multimodal analysis as described in any one of claims 1-5, characterized in that: The specific steps of this verification method are as follows: Ⅰ. Analyze various information in the paper and extract multimodal information in the paper. Based on the extracted multimodal information, build a corresponding paper analysis database; II. Identify and extract the experimental process in the paper, establish a standardized experimental protocol tree, and identify the academic ethical risks in the paper; III. After completing the method extraction and ethics review, create the corresponding experimental environment based on the generated experimental protocol tree, replicate the paper experiment, and evaluate the credibility of the experiment; IV. After the experimental replication is completed, the historical evolution path and controversial records of the paper’s viewpoints are tracked, and a spatiotemporal map of academic controversy is generated to analyze the controversial risks of the paper’s conclusions; V. Integrate real-time feedback from different sources, identify different types of feedback, distinguish rational criticism from emotional speech, and generate a credibility score for the paper's controversy; VI. Summarize all data from the verification process and generate a comprehensive verification report for the corresponding paper, while providing visual charts, dispute tracing path diagrams and experimental comparison results.

7. The full-process review intelligent verification method based on multimodal analysis according to claim 6 is characterized in that: The specific steps for reproducing the paper experiment described in Step III are as follows: S4.1: Parse the experimental method descriptions, mathematical formulas, and experimental configuration sections in the paper and extract the experimental operation steps, variable settings, model parameters, and hardware environment information. Calculate the dependency weights of each experimental operation step and organize them into a standardized experimental protocol tree based on the experimental process, variable dependencies, and control flow. S4.2: Expand each experimental step node from the constructed standardized experimental protocol tree, extract the complete execution path, and determine the operation sequence, input and output dependencies, and execution conditions of each node. Generate a corresponding operation sequence list. Then, map the input and output variables, control parameters, and hyperparameters in the standardized experimental protocol tree to variable names in the selected programming language, and establish binding relationships between variables and experimental operations. S4.3: Translate the algorithm descriptions, mathematical formulas, and pseudocode in the protocol tree into executable function modules in the program. During the translation process, the semantic consistency of each algorithm logic translation is calculated in real time. If the consistency falls below the preset threshold, it indicates a translation anomaly and the reviewer is requested to make manual adjustments. The corresponding function definition, loop, conditional judgment, and data processing flow are then generated according to the experimental logic. S4.4: Assemble the generated function modules, variable binding sections, pre-processing and post-processing processes into a complete executable script in the order of the execution path, and insert the required intermediate variable initialization, error handling statements, and log output modules to generate a complete experimental code; S4.5: Based on the programming language, operating system, and dependency framework involved in the experimental code, select the corresponding basic Docker container and initialize the container configuration. After initialization, parse the library dependency declarations in the experimental code and the dependency list extracted from the protocol tree, and install the corresponding dependency packages according to the dependency list. Then, automatically mount the experimental code, scripts, configuration files, and test data into the specified directory in the Docker container; S4.6: Automatically execute the generated experimental container in the Docker container, run the complete experimental process, extract the experimental result data, logs and output files generated in real time, and automatically compare the experimental data reported in the paper with the actual experimental results in the Docker container; S4.7: Based on the comparison results, evaluate the average difference between the actual experimental results and the experimental data reported in the paper, summarize the protocol tree structure, generated code, container configuration, run logs, and experimental comparison results, and generate a standardized reproducibility analysis report.

8. The full-process review intelligent verification method based on multimodal analysis according to claim 6 is characterized in that: The specific steps for analyzing the controversial risk of the paper conclusions described in Step IV are as follows: S5.1: Evaluate the subversiveness score of each conclusion paragraph, discussion paragraph, and abstract in the paper, and select sentences or paragraphs with subversiveness scores above a preset threshold. Then, search the historical paper database for historical viewpoints that correspond to or oppose the current viewpoint, and calculate the strength of the correlation between the current paper's viewpoint and the historical viewpoints. S5.2: Use historical viewpoints and current paper viewpoints as edges to build a semantic association network between papers. Annotate the inheritance, citation, or opposition relationships between viewpoints. Represent the relationships between papers as edges, using the strength of the association between each viewpoint as the weight of the corresponding edge. Collect discussions about the current paper viewpoint on different academic social platforms. S5.3: Extract the various viewpoints in the discussion content, including questions, defenses, and supplementary information, and mark the discussion time and user influence of each viewpoint. Calculate the social controversy heat of the current paper's viewpoint. Based on the viewpoint evolution data in historical documents and the academic social discussion timeline, calculate the propagation path span of the current paper's viewpoint, and generate the spatiotemporal evolution trajectory of the viewpoint. S5.4: Based on the generated spatiotemporal evolution trajectory, construct a time series diagram and a cross-institutional and cross-disciplinary communication path diagram. Use the publication time, geographic location, opinion dissemination channel, and the citation, inheritance, and opposition relationships between opinions as node data in the communication path diagram, and calculate the spatiotemporal density weight of each node data. S5.5: Based on the spatiotemporal density weights of each node data, distribute the nodes on the time axis and geographic coordinate system, and use different colors and shapes to mark the opinion type of each node. Then, use connecting lines to represent the association paths between opinions to generate a dispute map for the corresponding opinion. Calculate the dispute peak value for each area in each dispute map, and mark the area with a dispute peak value above a preset threshold as a dispute peak area; S5.6: Analyze the propagation paths of controversial viewpoints in each dispute peak area from their origin to the current state, and evaluate the complexity of the propagation paths. Based on the sum of the spatiotemporal density weights, the dispute peak, the complexity of the propagation paths, and the subversiveness of the initial viewpoints, calculate the dispute risk score of the current paper's viewpoints, and output a dispute risk assessment report.

Images