AIQ Human-Machine Collaboration Ability Analysis System Based on Dynamic Adaptation

Abstract

This invention relates to the field of artificial intelligence visual analysis technology, and discloses an AIQ human-machine collaborative ability analysis system based on dynamic adaptation. The system includes a three-dimensional parameter standardization definition and input module, a five-dimensional AIQ ability model matching and question bank selection module, a dynamic three-dimensional adaptive engine and assessment scale generation module, a multi-modal process data full-link acquisition module, a multi-dimensional data fusion analysis and report generation module, and a system self-iteration and parameter library update module. This invention achieves dynamic adaptive adjustment of the assessment system, featuring input and interaction design with both human-initiated input and machine-initiated capture. It can calibrate the assessment content, task difficulty, and scoring standards in real time according to the cognitive development stage of the test subject, the application scenario of the assessment, and the development level of artificial intelligence technology. This ensures a high degree of fit between the assessment and the test subject's situation, and avoids assessment failure problems caused by the iteration of artificial intelligence technology.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence visual analysis technology, and in particular to an AIQ human-machine collaborative capability analysis system based on dynamic adaptation. Background Technology

[0002] With the rapid popularization and application of artificial intelligence technology, especially large language models, multimodal intelligent systems and embodied intelligent robots, the ability of individuals to conduct efficient collaboration with artificial intelligence systems has become a key component of individuals' core competencies and professional competitiveness in the digital age. This ability is defined as human-machine collaborative literacy or artificial intelligence quotient. Current assessment schemes for this ability are mainly divided into two categories: traditional digital literacy knowledge assessment and fixed task performance assessment. Neither of these can achieve a comprehensive, accurate, and long-term assessment of an individual's real human-machine collaboration ability. They are also difficult to adapt to the current state of rapid iteration of artificial intelligence technology in the industry, as well as the differentiated assessment needs of different age groups and different application scenarios. Therefore, there is an urgent need for a systematic, dynamic, and multi-dimensional artificial intelligence quotient assessment technology solution. Summary of the Invention

[0003] The purpose of this invention is to provide an AIQ human-machine collaborative capability analysis system based on dynamic adaptation to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: The AIQ (AIQ-based) human-machine collaborative ability analysis system based on dynamic adaptation includes a three-dimensional parameter standardization definition and input module, a five-dimensional AIQ ability model matching and question bank screening module, a dynamic three-dimensional adaptive engine and assessment scale generation module, a multi-modal process data end-to-end acquisition module, a multi-dimensional data fusion analysis and report generation module, and a system self-iteration and parameter library update module. The output of the three-dimensional parameter standardization definition and input module is connected to the input of the five-dimensional AIQ ability model matching and question bank screening module. The output of the five-dimensional AIQ ability model matching and question bank screening module is connected to the input of the dynamic three-dimensional adaptive engine and assessment scale generation module. The output of the dynamic three-dimensional adaptive engine and assessment scale generation module is connected to the input of the multi-modal process data end-to-end acquisition module. The output of the multi-modal process data end-to-end acquisition module is connected to the input of the multi-dimensional data fusion analysis and report generation module. The output of the multi-dimensional data fusion analysis and report generation module is connected to the input of the system self-iteration and parameter library update module. The output of the system self-iteration and parameter library update module is connected to the input of the three-dimensional parameter standardization definition and input module.

[0005] As a further improvement to this technical solution: the three-dimensional parameter standardization definition and input module includes an age and cognitive development stage mapping submodule, an assessment scenario parameter configuration submodule, and an AI function benchmark dynamic update submodule. The age and cognitive development stage mapping submodule completes the standardized mapping from age values ​​to corresponding cognitive development stage labels, and stores metadata related to task complexity, knowledge background, and language expression level corresponding to each cognitive development stage. The assessment scenario parameter configuration submodule stores the ability dimension weight vectors corresponding to preset assessment scenarios, completes the matching of target scenarios and corresponding parameter calls in assessment requests, and the AI ​​function benchmark dynamic update submodule quantitatively collects the multimodal performance indicators of mainstream AI tools, establishes a regular update mechanism, and stores the AI ​​function benchmark values ​​corresponding to the assessment time.

[0006] As a further improvement to this technical solution: the three-dimensional parameter standardization definition and input module includes an age and cognitive development stage mapping submodule, an assessment scenario parameter configuration submodule, and an AI function benchmark dynamic update submodule. The age and cognitive development stage mapping submodule completes the standardized mapping from age values ​​to corresponding cognitive development stage labels, and stores metadata related to task complexity, knowledge background, and language expression level corresponding to each cognitive development stage. The assessment scenario parameter configuration submodule stores the ability dimension weight vectors corresponding to preset assessment scenarios, completes the matching of target scenarios and corresponding parameter calls in assessment requests, and the AI ​​function benchmark dynamic update submodule quantitatively collects the multimodal performance indicators of mainstream AI tools, establishes a regular update mechanism, and stores the AI ​​function benchmark values ​​corresponding to the assessment time.

[0007] As a further improvement to this technical solution: the dynamic 3D adaptive engine and assessment scale generation module includes a dynamic test paper matching submodule, an assessment interaction simulation environment configuration submodule, a dynamic scoring rule generation submodule, and a standardized scale output submodule. The dynamic test paper matching submodule completes the proportional allocation and extraction of assessment questions based on the input 3D parameters and the ability dimension weight vector corresponding to the scene. The core calculation logic of the dynamic test paper matching submodule adopts the following formula: , in the formula, for The number of items allocated to each dimension in this assessment scale. This refers to the total number of items in this assessment scale. for In the scene The weight values ​​corresponding to the dimensions These are the ordinal numbers for the five-dimensional capabilities, ranging from 1 to 5. For the target scenario tags corresponding to this assessment, the dynamic question-combination rule matching submodule calculates the number of questions in each dimension based on the formula, and extracts corresponding questions proportionally from the candidate question bank to form the basic framework of the assessment scale. The assessment interaction simulation environment configuration submodule configures the interactive environment for the assessment practice task based on the input AI function benchmark parameters, connects to real AI tool APIs that conform to the current AI function benchmark, or calls a pre-built AI behavior simulator to provide the test taker with an assessment interaction environment that matches the current AI technology conditions, achieving seamless integration between the assessment scale and the interaction environment. The dynamic scoring rule generation submodule generates appropriate scoring rules for each question in the scale based on the input AI function benchmark parameters. The core calculation logic of the dynamic scoring rule generation submodule adopts the following formula: , in the formula, for The scoring threshold for the question under the current AI functional benchmark. for The baseline scoring threshold for the question. for The question sets a baseline AI performance value. This refers to the current benchmark performance value of the AI ​​function corresponding to this evaluation. The dynamic scoring rule generation submodule generates a unique serial number for each item in the assessment scale. Based on the formula, it dynamically calibrates the scoring threshold for each item. The standardized scale output submodule completes the standardized integration and output of the assessment scale, integrating the item content, interactive environment entry, and scoring rules into a complete assessment execution file.

[0008] As a further improvement to this technical solution: the multimodal process data full-link acquisition module includes an interaction log acquisition submodule, a behavior time-series data acquisition submodule, and a process product retention submodule. The interaction log acquisition submodule records all data of the entire human-computer interaction process, including instruction input, AI response, and instruction correction and iteration. The behavior time-series data acquisition submodule records time-series data such as timestamps, response duration, and number of content modifications for key operations throughout the assessment process. The process product retention submodule stores all intermediate files, drafts, and staged outputs generated during the assessment process. The multimodal process data full-link acquisition module completes fine-grained acquisition of all behavioral data of the test subject throughout the entire assessment execution process and simultaneously acquires process data of the entire human-computer collaboration process.

[0009] As a further improvement to this technical solution: the multidimensional data fusion analysis and report generation module includes a dimension score calculation submodule, a process indicator mining submodule, and a visualization report generation submodule. The dimension score calculation submodule, based on dynamic scoring rules, completes the standardized score calculation of the five dimensions of the test subject's ability. The core calculation logic of the dimension score calculation submodule adopts the following formula: , in the formula, for The standardized score for each dimension for Dimension The actual score of the question, for The total number of questions corresponding to each dimension. The index is the serial number of the five-dimensional capability dimension, with a value range of 1 to 5. The process index mining submodule extracts the process behavior index corresponding to the five-dimensional capability dimension from the collected process data, and completes the quantitative extraction of the test subject's behavioral characteristics. The visualization report generation submodule integrates the dimension scores, process indicators, and norm benchmarking data to generate a standardized visualization assessment and diagnostic report.

[0010] As a further improvement to this technical solution: the system self-iteration and parameter library update module includes a group norm update submodule and a question bank and parameter library iteration submodule. The group norm update submodule completes the regular update of the corresponding age and scenario group norms based on the full set of evaluation data. The question bank and parameter library iteration submodule completes the adjustment of the question bank's question adaptation range and content iteration based on the AI ​​functional benchmark, eliminating invalid questions and supplementing with newly added adapted questions.

[0011] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention achieves dynamic adaptive adjustment of the assessment system, which can calibrate the assessment content, task difficulty and scoring standards in real time according to the cognitive development stage of the test subject, the application scenario of the assessment and the development level of artificial intelligence technology. This ensures a high degree of fit between the assessment and the test subject's situation, and avoids the assessment failure problem caused by the iteration of artificial intelligence technology. At the same time, through a multi-dimensional ability quantification system, it achieves comprehensive coverage and three-dimensional characterization of individual human-machine collaboration ability, which greatly improves the validity and accuracy of human-machine collaboration ability assessment.

[0012] 2. This invention achieves in-depth diagnosis of an individual's human-computer collaboration ability through fine-grained collection and in-depth analysis of process data throughout the entire human-computer interaction process. It can not only output quantitative results of ability, but also accurately locate the shortcomings and deficiencies of an individual's ability, providing clear guidance for subsequent ability improvement. At the same time, this solution has strong scenario adaptability and scalability. By adjusting the core parameters, it can adapt to the assessment needs of various scenarios such as education and vocational training, and has a wide range of applications. Meanwhile, the closed-loop self-iteration mechanism can ensure the long-term advanced nature and applicability of the assessment system.

[0013] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the method structure of the AIQ human-machine collaborative capability analysis system based on dynamic adaptation. Detailed Implementation

[0015] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are for illustrative purposes only and are not intended to limit the scope of the invention. The invention is described more specifically in the following paragraphs by way of example with reference to the accompanying drawings. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0016] Please see Figure 1 In this embodiment of the invention, the AIQ human-machine collaborative ability analysis system based on dynamic adaptation includes a three-dimensional parameter standardization definition and input module, a five-dimensional AIQ ability model matching and question bank screening module, a dynamic three-dimensional adaptive engine and assessment scale generation module, a multi-modal process data full-link acquisition module, a multi-dimensional data fusion analysis and report generation module, and a system self-iteration and parameter library update module. The output end of the three-dimensional parameter standardization definition and input module is connected to the input end of the five-dimensional AIQ ability model matching and question bank screening module. The output end of the five-dimensional AIQ ability model matching and question bank screening module is connected to the input end of the dynamic three-dimensional adaptive engine and assessment scale generation module. The output end of the dynamic three-dimensional adaptive engine and assessment scale generation module is connected to the input end of the multi-modal process data full-link acquisition module. The output end of the multi-modal process data full-link acquisition module is connected to the input end of the multi-dimensional data fusion analysis and report generation module. The output end of the multi-dimensional data fusion analysis and report generation module is connected to the input end of the system self-iteration and parameter library update module. The output end of the system self-iteration and parameter library update module is connected to the input end of the three-dimensional parameter standardization definition and input module. Specifically, the three-dimensional parameter standardization definition and input module is the system's input front end, responsible for receiving and standardizing the core basic parameters of the evaluation, providing a unified parameter benchmark for all subsequent processes; The five-dimensional AIQ ability model matching and question bank screening module is the core of the system's ability quantification. It is responsible for building a unified quantitative framework for human-machine collaborative ability and completing the preliminary screening of the assessment question bank based on input parameters, providing a compliant question basis for scale generation. The dynamic 3D adaptive engine and assessment scale generation module are the core execution units of the system. They are responsible for generating customized assessment scales, configuring interactive environments and calibrating scoring rules based on input parameters and the filtered question bank, and outputting a complete assessment solution that can be directly executed. The multimodal process data end-to-end acquisition module is the system's data acquisition unit, responsible for collecting all test takers' answers and process behavior data during the assessment process, providing complete data support for subsequent ability analysis; The multidimensional data fusion analysis and report generation module is the system's data analysis and output unit. It is responsible for performing multidimensional calculations and mining on the collected assessment data, generating standardized assessment and diagnostic reports, and outputting the quantitative results of the test subject's AIQ ability. The system self-iteration and parameter library update module is the closed-loop iterative unit of the system. It is responsible for updating and iterating the system's core parameter library, question bank, and norms based on the full set of evaluation data and the development of AI technology. It then sends the updated parameters back to the front-end input module to achieve dynamic adaptive updates of the system.

[0017] The three-dimensional parameter standardization definition and input module includes an age and cognitive development stage mapping submodule, an assessment scenario parameter configuration submodule, and an AI function benchmark dynamic update submodule. The age and cognitive development stage mapping submodule completes the standardized mapping from age values ​​to corresponding cognitive development stage labels, and stores metadata related to task complexity, knowledge background, and language expression level for each cognitive development stage. The assessment scenario parameter configuration submodule stores the ability dimension weight vectors corresponding to preset assessment scenarios, and completes the matching of target scenarios and corresponding parameter calls in assessment requests. The AI ​​function benchmark dynamic update submodule quantifies and collects multimodal performance indicators of mainstream AI tools, establishes a regular update mechanism, and stores the AI ​​function benchmark values ​​corresponding to the assessment time. Specifically, the age and cognitive development stage mapping submodule is the basic mapping unit within the module. It pre-establishes the correspondence between age values ​​and cognitive development stage labels. The cognitive development stage labels include seven categories: preschool, lower elementary, upper elementary, junior high, senior high, university, and professional adult. Each label has its own unique metadata, which includes the task complexity, knowledge background requirements, and language expression level requirements suitable for that cognitive stage. After receiving the age value of the test subject, the submodule automatically completes the matching of the corresponding cognitive development stage label and the retrieval of the corresponding metadata, providing age-dimensional parameter benchmarks for subsequent question bank screening and scale generation. The assessment scenario parameter configuration submodule is the scenario adaptation unit within the module. It pre-stores configuration parameters for multiple preset assessment scenarios, including four types: pre-admission assessment, pre-course assessment, job matching, and vocational skills diagnosis. Each scenario has a corresponding stored five-dimensional ability dimension weight vector. The weight vector limits the proportion and emphasis of each ability dimension in the assessment under the corresponding scenario. After receiving the target scenario of the assessment request, the submodule automatically completes the matching of the corresponding scenario and calls the weight vector, providing the scenario dimension parameter benchmark for subsequent test paper generation and scoring. The AI ​​Functional Benchmark Dynamic Update Submodule is the dynamic update unit within the module. It is responsible for quantifying and collecting the multimodal performance indicators of current mainstream AI tools. The performance indicators cover the ability to understand and generate text, voice, images, videos, actions, and environments, as well as quantitative values ​​such as execution accuracy, F1 score, and completion rate for specific domain tasks. The submodule establishes a fixed-cycle update mechanism to synchronize the latest AI technology performance data and store the AI ​​functional benchmark values ​​corresponding to the evaluation execution time. This provides AI technology-dimensional parameter benchmarks for subsequent question bank selection, environment configuration, and scoring rule calibration.

[0018] The five-dimensional AIQ ability model matching and question bank screening module includes a five-dimensional AIQ ability model storage submodule and a question bank initial screening and matching submodule. The five-dimensional AIQ ability model storage submodule solidifies the five-dimensional quantitative system of human-machine collaborative ability, decomposing human-machine collaborative ability into five dimensions: cognitive understanding, interactive collaboration, critical application, ethical awareness, and adaptive learning. Each dimension corresponds to a preset set of measurable behavioral indicators, and each behavioral indicator corresponds to a matching assessment question type. All assessment questions are bound to the corresponding dimensions and behavioral indicators, realizing a one-to-one correspondence between ability dimensions and assessment content. The question bank initial screening and matching submodule, based on the input three-dimensional parameters, completes the compliance screening of questions in the question bank and outputs a candidate question bank that is suitable for this assessment. Specifically, the five-dimensional AIQ capability model storage submodule is the core unit for capability quantification within the module. It solidifies the five-dimensional quantification system of human-machine collaboration capabilities, completely decomposing human-machine collaboration capabilities into five interrelated dimensions: cognitive understanding, interactive collaboration, critical application, ethical awareness, and adaptive learning. Each dimension has a corresponding set of measurable behavioral indicators. Among them, the behavioral indicators for the cognitive understanding dimension include the accuracy of describing the boundaries of AI capabilities and the depth of understanding of technical principles; the behavioral indicators for the interactive collaboration dimension include the clarity of initial instructions, the quality of optimized questioning in multi-turn dialogues, and the rationality of task decomposition and division of labor; the behavioral indicators for the critical application dimension include the error recognition rate in AI output. The evaluation criteria include: the ability to verify information sources and creatively integrate AI output with personal knowledge; ethical awareness indicators, including strategies for identifying and addressing issues such as data privacy, algorithmic bias, and copyright ownership; and adaptive learning indicators, including the efficiency of learning and using new AI tools and the ability to transfer existing collaborative strategies to new tasks. Each submodule matches a corresponding assessment question type for each behavioral indicator. The question types include scenario-based selection questions, task-based practical questions, and human-computer dialogue simulation questions. Every question in the full question bank is bound to a corresponding ability dimension and behavioral indicator, achieving a one-to-one correspondence between ability dimensions and assessment content, ensuring that the assessment questions accurately measure the ability level of the corresponding dimension. The question bank initial screening and matching submodule is the question selection execution unit within the module. It receives three core parameters—age, scenario, and AI function benchmark—from the standardized definition of three-dimensional parameters and the output of the input module. Simultaneously, it reads three parameters from the full question bank: the applicable age range, the suitable assessment scenario, and the required AI function prerequisites for each question. It executes a triple screening logic to select all valid questions whose applicable age range includes the current test subject's age, whose suitable assessment scenario matches or includes the current target scenario, and whose required AI function prerequisites are less than or equal to the current AI function benchmark value. Invalid questions that do not meet the parameter requirements are eliminated. Finally, it outputs a candidate question bank suitable for this assessment, providing a compliant question foundation for subsequent scale generation.

[0019] The dynamic 3D adaptive engine and assessment scale generation module includes a dynamic test paper matching submodule, an assessment interaction simulation environment configuration submodule, a dynamic scoring rule generation submodule, and a standardized scale output submodule. The dynamic test paper matching submodule completes the proportional allocation and extraction of assessment questions based on the input 3D parameters and the ability dimension weight vector corresponding to the scene. The core calculation logic of the dynamic test paper matching submodule adopts the following formula: , in the formula, for The number of items allocated to each dimension in this assessment scale. This refers to the total number of items in this assessment scale. for In the scene The weight values ​​corresponding to the dimensions These are the ordinal numbers for the five-dimensional capabilities, ranging from 1 to 5. For the target scenario tags corresponding to this assessment, the dynamic question-combination rule matching submodule calculates the number of questions in each dimension based on the formula, and extracts corresponding questions proportionally from the candidate question bank to form the basic framework of the assessment scale. The assessment interaction simulation environment configuration submodule configures the interactive environment for the assessment practice task based on the input AI function benchmark parameters, connects to real AI tool APIs that conform to the current AI function benchmark, or calls a pre-built AI behavior simulator to provide the test taker with an assessment interaction environment that matches the current AI technology conditions, achieving seamless integration between the assessment scale and the interaction environment. The dynamic scoring rule generation submodule generates appropriate scoring rules for each question in the scale based on the input AI function benchmark parameters. The core calculation logic of the dynamic scoring rule generation submodule adopts the following formula: , in the formula, for The scoring threshold for the question under the current AI functional benchmark. for The baseline scoring threshold for the question. for The question sets a baseline AI performance value. This refers to the current benchmark performance value of the AI ​​function corresponding to this evaluation. The dynamic scoring rule generation submodule generates a unique serial number for each item in the assessment scale. Based on the formula, it dynamically calibrates the scoring threshold for each item. The standardized scale output submodule completes the standardized integration and output of the assessment scale, integrating the item content, interactive environment entry, and scoring rules into a complete assessment execution file. Specifically, the dynamic test paper matching submodule is the test paper execution unit within the module. It is responsible for allocating the proportion of assessment questions and randomly selecting test papers based on the input three-dimensional parameters and the corresponding ability dimension weight vectors for the scenario. The core calculation logic of the submodule is implemented through a fixed formula, in which... The values ​​range from 1 to 5, corresponding to the five dimensions of cognitive understanding, interactive collaboration, critical application, ethical awareness, and adaptive learning, respectively. The sub-module first determines the total number of items in this assessment scale. Combined with the target scenario The weight values ​​corresponding to each dimension The number of questions to be allocated for each dimension is calculated using a formula. Then, from the candidate question bank, a corresponding number of questions are randomly selected and calculated for each dimension to form the basic framework of the assessment scale, ensuring that the distribution of questions in the scale is completely matched with the ability weights corresponding to the scenarios. The assessment interaction simulation environment configuration submodule is the interaction environment building unit within the module. It is responsible for configuring the interaction environment corresponding to the practical questions and human-computer dialogue simulation questions in the assessment scale based on the input AI functional benchmark parameters. The submodule can directly access real AI tool APIs that conform to the current AI functional benchmark, including large language model APIs, multimodal generation model APIs, robot control platforms, etc. In scenarios where real-time AI APIs cannot be accessed, the submodule can call a pre-built AI behavior simulator. The AI ​​behavior simulator, based on the performance data of the current AI functional benchmark, implements typical AI interaction mode responses through rule bases or lightweight models, providing test subjects with an assessment interaction environment that is fully matched with the current AI technology conditions. This achieves seamless integration between assessment scale questions and the interaction execution environment, ensuring the executability and authenticity of the assessment task. The dynamic scoring rule generation submodule is the scoring rule calibration unit within the module. It is responsible for generating appropriate scoring rules and thresholds for each item in the scale based on the input AI functional benchmark parameters. The core calculation logic of the submodule is implemented through a fixed formula, in which each item corresponds to a preset benchmark scoring threshold. Compared with benchmark AI functional performance values The sub-modules are combined with the current AI function benchmark performance values ​​corresponding to this evaluation. The actual scoring threshold for this question in this assessment is calculated using a formula. This allows for the dynamic calibration of the scoring threshold for each question; as AI technology improves, Greater than When the AI ​​technology is in a certain state, the scoring threshold calculated by the formula will increase accordingly, thus raising the required ability level for the test taker; when the AI ​​technology capability declines, Less than At the same time, the scoring threshold will be lowered to ensure the differentiation and accuracy of the assessment and to avoid the assessment task losing its differentiation due to the development of AI technology; The standardized scale output submodule is the scale integration and output unit within the module. It is responsible for completing the standardized integration and final output of the assessment scale. It integrates the question content generated by the dynamic test paper generation, the interactive environment entry generated by the assessment interactive simulation environment configuration submodule, and the scoring details and thresholds generated by the dynamic scoring rule generation submodule into a complete and directly executable assessment execution file, which is then output to the subsequent multimodal process data end-to-end acquisition module.

[0020] The multimodal process data full-link acquisition module includes an interaction log acquisition submodule, a behavior time-series data acquisition submodule, and a process product retention submodule. The interaction log acquisition submodule records all data from the entire human-computer interaction process, including instruction input, AI response, and instruction correction and iteration. The behavior time-series data acquisition submodule records time-series data such as timestamps, response duration, and number of content modifications for key operations throughout the assessment process. The process product retention submodule stores all intermediate files, drafts, and staged outputs generated during the assessment process. The multimodal process data full-link acquisition module completes fine-grained acquisition of all behavioral data of the test subjects throughout the entire assessment execution process and simultaneously collects process data from the entire human-computer collaboration process. Specifically, the interaction log collection submodule is the interaction data collection unit within the module. It is responsible for recording all the human-computer interaction data between the test subject and the AI ​​system throughout the entire evaluation process. This includes the content of each instruction input by the test subject, the corresponding response content of the AI ​​system, the test subject's correction and iteration process of the instructions, and the number of rounds and time intervals of human-computer dialogue. It fully reconstructs the entire interaction process between the test subject and the AI ​​system, providing basic data for subsequent analysis of the test subject's interactive collaboration capabilities. The behavioral time-series data collection submodule is the behavioral data collection unit within the module. It is responsible for recording the time-series data of all key operations of the test taker throughout the entire assessment process. This includes the timestamps of key operations, the duration of hesitation in answering single questions and tasks, the number of times the answer content is modified, and the number and time points of operation withdrawal. By capturing the behavioral characteristics of the test taker during the assessment process through time-series data, it provides time-series data support for subsequent analysis of the test taker's ability shortcomings and behavioral habits. The Process Product Retention Submodule is the process document collection unit within the module. It is responsible for storing all process documents generated by the test subject during the entire evaluation process, including intermediate code, draft documents, charts, and phased task outputs. It fully retains the outputs of the test subject throughout the entire process of completing the evaluation task, not limited to the final task result. This provides complete process product data support for subsequent analysis of the test subject's task decomposition ability, content integration ability, and critical application ability. The multimodal process data end-to-end acquisition module simultaneously records the final response of the test taker and collects three types of process data in a fine-grained manner throughout the entire assessment process. This enables full-dimensional and end-to-end data coverage of the test taker's human-machine collaborative behavior, providing a complete and multi-dimensional dataset for subsequent capability analysis.

[0021] The multidimensional data fusion analysis and report generation module includes a dimension score calculation submodule, a process indicator mining submodule, and a visualization report generation submodule. The dimension score calculation submodule, based on dynamic scoring rules, completes the standardized score calculation of the test subject's five-dimensional ability dimensions. The core calculation logic of the dimension score calculation submodule adopts the following formula: , in the formula, for The standardized score for each dimension for Dimension The actual score of the question, for The total number of questions corresponding to each dimension. The index is the serial number of the five-dimensional ability dimension, with a value range of 1 to 5. The process index mining submodule extracts the process behavior index of the corresponding five-dimensional ability dimension from the collected process data to complete the quantitative extraction of the test subject's behavior characteristics. The visualization report generation submodule integrates the dimension scores, process indicators, and norm benchmarking data to generate a standardized visualization assessment and diagnostic report. Specifically, the dimensional score calculation submodule is the basic score calculation unit within the module. It is responsible for generating the scoring details and thresholds output by the submodule based on dynamic scoring rules, completing the standardized score calculation for the test subject's five-dimensional ability dimensions. The core calculation logic of the submodule is implemented through a fixed formula, in which... The values ​​range from 1 to 5, corresponding to five dimensions: cognitive understanding, interactive collaboration, critical application, ethical awareness, and adaptive learning. The submodule first counts the test subjects' scores on these dimensions. The actual score for each question under each dimension Then sum the actual scores of all questions under that dimension and divide by the total number of questions corresponding to that dimension. Calculate the standardized final score for this dimension. The scores of the five dimensions are calculated sequentially to form a standardized five-dimensional ability score matrix for the test subject, providing basic quantitative score data for subsequent ability diagnosis. The process indicator mining submodule is a deep behavior analysis unit within the module. It is responsible for extracting process behavior indicators corresponding to the five-dimensional ability dimensions from the process data output by the multimodal process data full-link acquisition module, and completing the quantitative extraction of the test subject's behavioral characteristics. The extractable process behavior indicators include the percentage improvement in AI output quality per instruction optimization, the response speed for identifying AI output errors, the response speed for identifying potential ethical issues, the exploration efficiency of new AI tools, and the migration and adaptation of collaborative strategies. Through in-depth mining of process data, the submodule achieves in-depth quantitative analysis of the test subject's human-machine collaboration ability, not limited to the final question score, but also locating the deep behavioral characteristics of the test subject's ability performance. The visualization report generation submodule is the report output unit within the module. It is responsible for integrating the standardized five-dimensional ability scores output by the dimensional score calculation submodule, the process behavior indicators output by the process indicator mining submodule, and the group norm data corresponding to age and scenario. It completes the ability benchmarking analysis of the test subject and generates a standardized visualization assessment and diagnostic report. The report content includes a five-dimensional ability radar chart, an ability development profile, the ranking of comprehensive ability in the corresponding group, detailed diagnosis of sub-ability, and personalized ability improvement suggestions. It fully presents the test subject's AIQ human-machine collaboration ability profile and provides the test subject with clear and actionable ability diagnosis results.

[0022] The system's self-iteration and parameter library update module includes a group norm update submodule and a question bank and parameter library iteration submodule. The group norm update submodule is based on the full set of evaluation data and completes the regular updates of the corresponding age and scenario group norms. The question bank and parameter library iteration submodule is based on the AI ​​functional benchmark and completes the adjustment of the question bank's scope of adaptation and content iteration, eliminating invalid questions and supplementing with newly added adapted questions. Specifically, the group norm update submodule is the norm iteration unit within the module. It is responsible for receiving the full set of assessment data output by the multidimensional data fusion analysis and report generation module, classifying and statistically analyzing it according to the age, cognitive development stage, and assessment scenario of the test subjects, and completing the regular updates of the group norms for the corresponding age and scenario. The group norms are the AIQ ability score distribution data of the corresponding group, which serve as the benchmark for the ability benchmarking analysis of the test subjects. By regularly updating the norm data, the submodule ensures the timeliness and representativeness of the norm data, and ensures the accuracy of the assessment benchmarking analysis. The question bank and parameter library iteration submodule is the question bank and parameter iteration unit within the module. It is responsible for receiving the latest AI function benchmark data output by the AI ​​function benchmark dynamic update submodule, and completing the adjustment of the adaptation range and content iteration of questions in the full question bank. The submodule will verify the required AI function prerequisites for each question, eliminate invalid questions that have lost their discriminatory power due to the development of AI technology, and at the same time supplement new adapted questions that conform to the latest AI function benchmark and are suitable for various ages and scenarios. It will also update the standardized definition of three-dimensional parameters and the core parameter library in the input module, and send the updated parameters back to the front-end input module to realize the dynamic adaptive iteration of the system, ensuring that the assessment system is always adapted to the current level of AI technology development and maintaining the effectiveness and discriminatory power of the assessment.

[0023] The method of use and working principle of this invention are as follows: Usage: Upon receiving an assessment request, the system first standardizes and inputs three core parameters—the test subject's cognitive development stage, the assessment target scenario, and the current AI functional benchmark—through the three-dimensional parameter standardization definition and input module. Then, through the five-dimensional AIQ capability model matching and question bank screening module, it completes compliance screening of the assessment question bank based on the input core parameters, generating a candidate question bank suitable for this assessment. Subsequently, through the dynamic three-dimensional adaptive engine and assessment scale generation module, it completes the assembly of the assessment scale, configuration of the assessment interaction environment, and generation of dynamic scoring rules, outputting an executable standardized assessment scale. During the assessment process in the generated interactive environment, the multimodal process data end-to-end acquisition module collects and retains assessment results and process data throughout the entire process. Then, through the multi-dimensional data fusion analysis and report generation module, it performs multi-dimensional calculations, in-depth indicator mining, and norm benchmarking analysis of the assessment data, outputting a visualized assessment diagnostic report. Finally, through the system self-iteration and parameter library update module, it regularly updates and iterates the group norms, question bank, and core parameter library, achieving long-term adaptive optimization of the system.

[0024] Working Principle: Based on three-dimensional dynamic adaptive logic, the system uses the cognitive development stage of the test subject, the assessment application scenario, and the current level of artificial intelligence technology as three core independent variables. It constructs a five-dimensional quantitative model covering all dimensions of human-machine collaboration capabilities as the core framework of the assessment. Through a dynamic engine, it completes the selection of assessment questions, test paper generation, and dynamic calibration of scoring rules based on the three independent variables. This ensures that the assessment content, difficulty, and scoring standards are highly matched with the assessment subjects, application scenarios, and current technical conditions. At the same time, it uses multimodal data acquisition technology to fully capture the behavioral data of the entire human-machine collaboration process. It is not limited to the final result but also performs multi-dimensional quantitative analysis and in-depth diagnosis of individual capabilities. Finally, through a closed-loop iterative mechanism, it synchronizes the development of artificial intelligence technology and group assessment data to complete the system self-optimization, continuously ensuring the accuracy, discrimination, and timeliness of the assessment.

[0025] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Those skilled in the art can readily implement the present invention based on the description and drawings above. However, any modifications, alterations, and variations made by those skilled in the art without departing from the scope of the present invention using the disclosed technical content are equivalent embodiments of the present invention. Furthermore, any modifications, alterations, and variations made to the above embodiments based on the essential technology of the present invention are still within the protection scope of the present invention.

Claims

1. A dynamic adaptive AIQ human-machine collaborative capability analysis system, characterized in that, The system includes a 3D parameter standardization definition and input module, a 5D AIQ ability model matching and question bank selection module, a dynamic 3D adaptive engine and assessment scale generation module, a multimodal process data end-to-end acquisition module, a multidimensional data fusion analysis and report generation module, and a system self-iteration and parameter library update module. The output of the 3D parameter standardization definition and input module is connected to the input of the 5D AIQ ability model matching and question bank selection module. The output of the 5D AIQ ability model matching and question bank selection module is connected to the input of the dynamic 3D adaptive engine and assessment scale generation module. The output of the dynamic 3D adaptive engine and assessment scale generation module is connected to the input of the multimodal process data end-to-end acquisition module. The output of the multimodal process data end-to-end acquisition module is connected to the input of the multidimensional data fusion analysis and report generation module. The output of the multidimensional data fusion analysis and report generation module is connected to the input of the system self-iteration and parameter library update module. The output of the system self-iteration and parameter library update module is connected to the input of the 3D parameter standardization definition and input module.

2. The AIQ human-machine collaboration capability analysis system based on dynamic adaptation according to claim 1, characterized in that, The three-dimensional parameter standardization definition and input module includes an age and cognitive development stage mapping submodule, an assessment scenario parameter configuration submodule, and an AI function benchmark dynamic update submodule. The age and cognitive development stage mapping submodule completes the standardized mapping from age values ​​to corresponding cognitive development stage labels, and stores metadata related to task complexity, knowledge background, and language expression level corresponding to each cognitive development stage. The assessment scenario parameter configuration submodule stores the ability dimension weight vectors corresponding to preset assessment scenarios, and completes the matching of target scenarios and corresponding parameter calls in assessment requests. The AI ​​function benchmark dynamic update submodule quantifies and collects multimodal performance indicators of mainstream AI tools, establishes a regular update mechanism, and stores the AI ​​function benchmark values ​​corresponding to the assessment time.

3. The AIQ human-machine collaboration capability analysis system based on dynamic adaptation according to claim 1, characterized in that, The five-dimensional AIQ ability model matching and question bank screening module includes a five-dimensional AIQ ability model storage submodule and a question bank initial screening and matching submodule. The five-dimensional AIQ ability model storage submodule solidifies the five-dimensional quantitative system of human-machine collaborative ability, decomposing human-machine collaborative ability into five dimensions: cognitive understanding, interactive collaboration, critical application, ethical awareness, and adaptive learning. Each dimension corresponds to a preset set of measurable behavioral indicators, and each behavioral indicator corresponds to a matching assessment question type. All assessment questions are bound to the corresponding dimensions and behavioral indicators, realizing a one-to-one correspondence between ability dimensions and assessment content. The question bank initial screening and matching submodule, based on the input three-dimensional parameters, completes the compliance screening of questions in the question bank and outputs a candidate question bank suitable for this assessment.

4. The AIQ human-machine collaboration capability analysis system based on dynamic adaptation according to claim 1, characterized in that, The dynamic 3D adaptive engine and assessment scale generation module include a dynamic test paper matching submodule, an assessment interaction simulation environment configuration submodule, a dynamic scoring rule generation submodule, and a standardized scale output submodule. The dynamic test paper matching submodule completes the proportional allocation and extraction of assessment questions based on the input 3D parameters and the ability dimension weight vector corresponding to the scene. The core calculation logic of the dynamic test paper matching submodule adopts the following formula: , in the formula, for The number of items allocated to each dimension in this assessment scale. This refers to the total number of items in this assessment scale. for In the scene The weight values ​​corresponding to the dimensions These are the ordinal numbers for the five-dimensional capabilities, ranging from 1 to 5. For the target scenario tags corresponding to this assessment, the dynamic question-combination rule matching submodule calculates the number of questions in each dimension based on the formula, and extracts corresponding questions proportionally from the candidate question bank to form the basic framework of the assessment scale. The assessment interaction simulation environment configuration submodule configures the interactive environment for the assessment practice task based on the input AI function benchmark parameters, connects to real AI tool APIs that conform to the current AI function benchmark, or calls a pre-built AI behavior simulator to provide the test taker with an assessment interaction environment that matches the current AI technology conditions, achieving seamless integration between the assessment scale and the interaction environment. The dynamic scoring rule generation submodule generates appropriate scoring rules for each question in the scale based on the input AI function benchmark parameters. The core calculation logic of the dynamic scoring rule generation submodule adopts the following formula: , in the formula, for The scoring threshold for the question under the current AI functional benchmark. for The baseline scoring threshold for the question. for The question sets a baseline AI performance value. This refers to the current benchmark performance value of the AI ​​function corresponding to this evaluation. The dynamic scoring rule generation submodule generates a unique serial number for each item in the assessment scale. Based on the formula, it dynamically calibrates the scoring threshold for each item. The standardized scale output submodule completes the standardized integration and output of the assessment scale, integrating the item content, interactive environment entry, and scoring rules into a complete assessment execution file.

5. The AIQ human-machine collaboration capability analysis system based on dynamic adaptation according to claim 1, characterized in that, The multimodal process data full-link acquisition module includes an interaction log acquisition submodule, a behavior time-series data acquisition submodule, and a process product retention submodule. The interaction log acquisition submodule records all data of the entire human-computer interaction process, including instruction input, AI response, and instruction correction and iteration. The behavior time-series data acquisition submodule records time-series data such as timestamps, response duration, and number of content modifications for key operations throughout the assessment process. The process product retention submodule stores all intermediate files, drafts, and staged outputs generated during the assessment process. The multimodal process data full-link acquisition module completes fine-grained acquisition of all behavioral data of the test subjects throughout the entire assessment execution process and simultaneously collects process data of the entire human-computer collaboration process.

6. The AIQ human-machine collaboration capability analysis system based on dynamic adaptation according to claim 1, characterized in that, The multidimensional data fusion analysis and report generation module includes a dimension score calculation submodule, a process indicator mining submodule, and a visualization report generation submodule. The dimension score calculation submodule, based on dynamic scoring rules, completes the standardized score calculation of the five dimensions of the test subject's ability. The core calculation logic of the dimension score calculation submodule adopts the following formula: , in the formula, for The standardized score of the dimensions for Dimension The actual score of the question for The total number of questions corresponding to each dimension. The index is the serial number of the five-dimensional capability dimension, with a value range of 1 to 5. The process index mining submodule extracts the process behavior index corresponding to the five-dimensional capability dimension from the collected process data, and completes the quantitative extraction of the test subject's behavioral characteristics. The visualization report generation submodule integrates the dimension scores, process indicators, and norm benchmarking data to generate a standardized visualization assessment and diagnostic report.

7. The AIQ human-machine collaboration capability analysis system based on dynamic adaptation according to claim 1, characterized in that, The system's self-iteration and parameter library update module includes a group norm update submodule and a question bank and parameter library iteration submodule. The group norm update submodule updates the corresponding age and scenario group norms regularly based on the full set of evaluation data. The question bank and parameter library iteration submodule updates the question bank based on the AI ​​functional benchmark, adjusts the scope of question adaptation and iterates the content, eliminates invalid questions and adds new adapted questions.

Images