Method and system for testing collaborative learning ability of intelligent mining equipment

Abstract

The invention discloses a method and a system for testing collaborative learning ability of intelligent mining equipment. According to the method, a composite mining task model containing environment uncertainty parameters is constructed in a simulation experiment environment, multi-mode demonstration data in the process that a human expert operator completes the task are collected, and simulation learning training is executed on a control model of intelligent mining equipment based on the demonstration data. And obtaining quantitative result data representing the equipment skill learning efficiency through an incremental'demonstration-learning-testing 'process. And on the basis, controlling the learned equipment model and the green operator to execute a man-machine collaborative test process based on structured interpretation data, and synchronously acquiring operation performance data, man-machine interaction event data, physiological state data and numerical man-machine interaction subjective feedback parameter data. And carrying out fusion processing and index calculation on the multi-source data to generate quantitative result data representing the collaborative teaching ability. And finally, integrating the skill learning efficiency result data and the collaborative teaching ability result data to form a multi-level and quantitative evaluation result of the man-machine collaborative learning ability of the intelligent mining equipment. According to the invention, systematic evaluation of learning efficiency, skill generalization ability and man-machine collaborative operation performance of intelligent mining equipment can be realized, and the method has strong repeatability, objectivity and versatility.

Description

Technical Field

[0001] This invention relates to the field of intelligent mining equipment technology, and in particular to a method and system for testing the collaborative learning ability of intelligent mining equipment. Background Technology

[0002] With the development of intelligent manufacturing and artificial intelligence technologies, intelligent mining equipment is increasingly being applied to complex operational scenarios such as mines. Unlike traditional manually operated equipment, it relies on control models or algorithms for autonomous or semi-autonomous operation. Its operational efficiency and safety depend on the learning ability of the control model and its level of collaboration with human operators. Existing technologies for testing and evaluating the performance of intelligent mining equipment often focus on single performance indicators, frequently employing offline testing or static scenario comparison experiments. While these evaluation methods can reflect the equipment's performance under fixed conditions, they struggle to characterize the process by which it learns and masters skills from human demonstrations, and they also fail to reflect its ability to generalize skills in complex and changing environments. In recent years, data-driven methods based on human demonstrations, such as imitation learning, have been widely applied in the training of intelligent equipment control models. Related research shows that introducing operational demonstration data from human experts can reduce the training cost of control models and accelerate learning convergence to some extent. However, current evaluations of the effectiveness of imitation learning largely rely on comparisons of single or limited test results, lacking a systematic and quantitative evaluation method for the efficiency of the learning process, learning convergence conditions, and the efficiency of utilizing demonstration data.

[0003] On the other hand, in actual operations, intelligent mining equipment often needs to work collaboratively with human operators. For example, in semi-autonomous control mode, the equipment performs the main operational actions, while the operator intervenes or takes over at critical points. Current technologies for evaluating the effectiveness of human-machine collaborative operations often rely on the operator's experience or simple statistics on operational success rates. They lack a systematic evaluation method that integrates and analyzes operational performance data, human-machine interaction data, and objective physiological state data, making it difficult to comprehensively reflect the stability of the interaction between equipment and operator and the teaching effectiveness during collaborative operations.

[0004] Furthermore, some intelligent equipment possesses the ability to output information about its internal decision-making processes. How to evaluate the effectiveness of such interaction mechanisms in actual collaborative operations is a pressing issue. Existing evaluation methods are mostly qualitative analyses or based on single subjective feedback indicators, lacking standardized, reproducible testing procedures and quantitative indicator systems, making it difficult to conduct horizontal comparisons of different equipment, control models, or interaction mechanisms. Therefore, existing technologies still lack a testing method and system that can comprehensively consider demonstration learning efficiency, skill generalization ability, and human-machine collaborative operation performance in a unified testing environment to conduct multi-dimensional and quantitative evaluation of the human-machine collaborative learning ability of intelligent mining equipment. Summary of the Invention

[0005] This invention aims to overcome the shortcomings of existing technologies and provide a method and system for testing the collaborative learning capabilities of intelligent mining equipment. Traditional intelligent equipment testing methods primarily focus on the automation accuracy and reliability of the equipment, or the operator's proficiency in fixed modes. They lack systematic and quantitative means to evaluate the equipment's ability to learn from human experience and its capacity to enhance human-machine trust and collaborative efficiency through interpretable behavior. This makes it impossible to scientifically measure and compare the "collaborative intelligence" levels of different intelligent equipment, hindering the development and application of advanced human-machine integration technologies. Therefore, the objectives of this invention include: 1. To provide a method and system that can quantitatively evaluate the efficiency of intelligent mining equipment in acquiring human expert skills through imitation learning.

[0006] 2. A method and system are provided that can quantitatively evaluate the structured interpretable data generated by intelligent mining equipment based on the internal decision-making state of the control model, support the interactive data collection and processing in the process of human-machine collaborative operation, and thereby improve the operational performance of human-machine collaborative operation related results data.

[0007] 3. Construct a closed-loop, two-way, data-driven standardized testing framework to transform an interactive test into an evaluation process of "human-machine co-evolution," providing objective basis for equipment R&D optimization, selection and acceptance, and operator collaborative training.

[0008] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: A method for testing the collaborative learning ability of intelligent mining equipment, characterized in that the testing method includes the following steps: S1. Construct a composite mining task model with environmental uncertainty parameters and autonomous decision-making constraints in a simulated experimental environment, and initialize the control model of the intelligent mining equipment to be tested.

[0009] S2. Collect multimodal demonstration data of human expert operators completing composite mining tasks, conduct imitation learning training or parameter fine-tuning of the control model, and obtain equipment skill learning efficiency results data through an incremental demonstration-learning-test process, including: S21. Collect multimodal demonstration data, including operation instruction sequences, environmental conditions and expert attention focus related data.

[0010] S22. Call the multimodal demonstration data to train or fine-tune the control model and generate an updated control model.

[0011] S23. Use the updated control model to execute the task in a preset standard test scenario and collect performance data, including task completion time, resource consumption, and job quality parameters.

[0012] S24. Generate skill learning efficiency results data based on performance data, including the number of demonstrations required for learning convergence, skill generalization ability score, or a combination of both.

[0013] S3. The intelligent mining equipment will execute a human-machine collaborative testing process based on structured interpreted data. By collecting and processing human-machine interaction data, test result data will be generated to characterize the collaborative teaching effectiveness of the equipment, including: S31. During equipment operation, the internal decision-making state data based on the equipment control model generates and outputs structured explanatory data corresponding to key decision-making behaviors.

[0014] S32. After outputting the interpretation data, allow novice operators to work collaboratively with the equipment or take over the operation, and collect operational performance and human-machine interaction event data.

[0015] S33. Collect physiological state data and subjective feedback information of novice operators, and convert the subjective feedback information into numerical trust parameter data.

[0016] S34. Generate collaborative teaching ability result data based on the above data, including operational performance improvement rate, human-computer conflict reduction rate, human-computer interaction subjective feedback change value or a combination thereof.

[0017] S4. Based on the skill learning efficiency results data and the multi-source data collected during the collaborative testing process, perform fusion processing and index calculation to generate a multi-level, quantitative evaluation result of the human-machine collaborative learning capability of intelligent mining equipment. Furthermore, the number of demonstrations required for learning convergence is determined through the following process: In a simulated experimental environment, based on the core performance data of multiple expert operators completing the same target task, the average value μ and standard deviation σ of the expert group on each performance index are calculated, and combined with a preset tolerance coefficient α, the expert performance baseline threshold for judging learning convergence is determined; In the process of incrementally introducing expert demonstration data, the intelligent mining equipment control model is repeatedly subjected to demonstration learning training and autonomous operation testing under standard test scenarios, and the model performance data obtained from the test is compared with the expert performance baseline threshold. When the average performance of the model on each core performance index reaches or exceeds the corresponding threshold, the model learning convergence is determined, and the cumulative number of expert demonstrations used at this time is determined as the number of demonstrations required for learning convergence. Furthermore, the tolerance coefficient α is a preset positive number used to adjust the stringency of the evaluation criteria. A larger α value indicates that the performance of the equipment model is required to be closer to the high-end level of the expert group, and the evaluation criteria are more stringent; a smaller α value indicates a more lenient evaluation criteria. Based on statistical experience in engineering testing and the requirements for evaluation stability, the typical range of α is between 0.5 and 1.5.

[0018] Furthermore, the skill generalization ability score is determined as follows: At least one generalization test scenario of the same task type as the training demonstration standard scenario but with preset differences is constructed in a simulated experimental environment. The intelligent mining equipment control model, which has already learned and converged in the standard scenario, performs multiple autonomous operations to obtain the model's average performance value. A performance evaluation benchmark value is determined based on expert operation results or preset theoretical performance standards, and a corresponding skill generalization ability score is generated based on the comparison between the model's average performance value and the benchmark value. When multiple generalization test scenarios exist, the skill generalization ability scores of each scenario are fused to generate the final comprehensive skill generalization ability score. Furthermore, the operational performance improvement rate is determined by comparing the performance data of novice operators completing the same or equivalent task in unassisted mode and assisted mode. The performance data includes at least task completion time and operation accuracy. Based on the performance data, the task efficiency improvement rate and operation accuracy improvement rate are calculated respectively. The operational performance improvement rate is a combination result or a weighted composite result of the two.

[0019] Furthermore, the reduction rate of human-machine conflict is determined by statistically analyzing the number of human-machine conflict events that occur during the execution of tasks by intelligent mining equipment in both unassisted and assisted modes. These human-machine conflict events include at least emergency manual overwriting, forced interruption of automatic processes, and mutually exclusive operation command events. The reduction rate of human-machine conflict is calculated based on the total number of conflict events in unassisted mode and the total number of conflict events in assisted mode.

[0020] Furthermore, the change value of the human-computer interaction subjective feedback is determined by comparing the quantitative subjective feedback data obtained by the operator after the unassisted test and the assisted test, and the change value of the feedback is used as one of the quantitative parameters for the assessment of collaborative teaching ability.

[0021] This invention provides an intelligent mining equipment collaborative learning capability testing system, the system comprising: The task and test management module is used to build composite mining task models and schedule test processes; The data acquisition module is used to collect and store operation instructions, environmental status, task execution performance, human-computer interaction events, physiological status and subjective feedback data; The model processing module performs imitation learning training or parameter fine-tuning on the intelligent mining equipment control model based on multimodal demonstration data, and controls the updated model to execute composite mining tasks. The explanation and interaction module is used to generate and output structured interactive information based on the internal decision-making state data of the control model; the evaluation and reporting module is used to perform fusion processing and index calculation on multi-source data to generate skill learning efficiency, collaborative teaching ability and comprehensive evaluation results.

[0022] The modules are connected through a communication interface and operate collaboratively under the unified scheduling of the processor. The present invention also provides a testing system for the collaborative learning ability of intelligent mining equipment. The system is deployed on a computer device or server and is used to control the intelligent mining equipment to execute the test process in a simulated experimental environment, and to collect, process and analyze multi-source data generated during the test to generate test result data for evaluating human-machine collaborative learning ability.

[0023] The system includes at least: a task and test management module, a data acquisition module, a model processing module, an interpretation and interaction module, and an evaluation and reporting module. Each module is implemented by the processor executing program instructions stored in the memory.

[0024] The task and test management module is used to build a composite mining task model in a simulated experimental environment, and to uniformly schedule and manage each functional module according to a preset test process. The model includes environmental uncertainty parameters, autonomous decision-making constraints, and task objective parameters.

[0025] The data acquisition module collects and records multi-source test data during testing, including data on operation commands, environmental status, and task execution performance, and stores it in memory.

[0026] The model processing module calls the multimodal demonstration data in the memory to conduct imitation learning training or parameter fine-tuning on the intelligent mining equipment control model, and controls the updated model to perform composite mining tasks in a preset scenario.

[0027] The explanation and interaction module generates and outputs structured explanation data corresponding to key decision behaviors based on the decision state data inside the control model, and supports human-computer collaborative testing and interaction.

[0028] The assessment and reporting module processes and calculates indicators based on the collected data, generating data on skill learning efficiency and collaborative teaching ability, and based on this, generates a multi-level, quantitative comprehensive assessment report. The working principle and beneficial effects of this invention are as follows: 1. A quantitative assessment of the learning ability of intelligent mining equipment has been achieved. This invention introduces indicators such as "number of demonstrations required for learning convergence" and "skill generalization ability score," avoiding evaluation methods that rely solely on single performance tests or subjective judgments. This allows for quantitative analysis of the equipment's imitation learning efficiency and skill transfer ability in repeatable and comparable numerical forms.

[0029] 2. Improved the objectivity and stability of human-computer collaboration test results. By jointly analyzing operational performance data, human-computer interaction event data, and physiological state data, and by using physiological load indicators as weighting or correction factors in the evaluation calculation, the impact of single subjective feedback on the evaluation results was reduced, thus improving the objectivity and robustness of collaborative teaching ability evaluation.

[0030] 3. Supports systematic verification of the effectiveness of interpretive human-computer interaction mechanisms. This invention introduces a structured interpretive data output mechanism during collaborative testing, enabling key decision states of the equipment control model to participate in human-computer interaction in a standardized data format. By comparing changes in operational performance under unassisted and assisted testing, the impact of interpretive interaction mechanisms on collaborative operation effectiveness is verified.

[0031] 4. Excellent versatility and scalability. The task configuration method, evaluation index calculation method, and modular system structure adopted in this invention are not dependent on specific mining processes or single equipment types. They can be adapted to different types of intelligent mining equipment and different operational scenarios, and have high engineering application value.

[0032] 5. It facilitates the optimization and iteration of intelligent equipment control models. By systematically outputting skill learning efficiency results data and collaborative teaching ability results data, this invention can provide quantitative basis for the selection of training strategies, parameter tuning, and human-computer interaction mechanism design for control models, thereby supporting the continuous optimization of intelligent mining equipment. Attached Figure Description

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0034] Figure 1 This invention provides a schematic flowchart of a method for testing the collaborative learning ability of intelligent mining equipment. Figure 2 This invention provides a structural schematic diagram of an intelligent mining equipment collaborative learning capability testing system. Detailed Implementation

[0035] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0036] like Figure 1As shown, the present invention provides a method for testing the collaborative learning ability of intelligent mining equipment. The method includes: S1. In a simulated experimental environment, a composite mining task model containing environmental uncertainty parameters and autonomous decision-making constraints is constructed by the task and test management module.

[0037] S2. The task and test management module controls the intelligent mining equipment and its control model to execute a demonstration learning test process in a simulated experimental environment to generate test result data to characterize the equipment's skill learning effect. The demonstration learning test process includes at least the following steps: S21. The data acquisition module collects and records multimodal demonstration data during the completion of the composite mining task by one or more human expert operators. The multimodal demonstration data includes at least operation instruction sequence data, environmental state data, and expert attention focus-related data collected through biosensor devices. The multimodal demonstration data is stored in a memory. S22. The model processing module calls the stored multimodal demonstration data to perform imitation learning training or parameter fine-tuning on the control model of the intelligent mining equipment to generate an updated equipment control model. S23. The model processing module controls the updated equipment control model to autonomously execute the composite mining task in a preset standard test scenario. The data acquisition module synchronously collects performance data during the task execution process. The performance data includes at least task completion time, resource consumption, and operation quality parameters. S24. The evaluation and reporting module performs data processing operations based on the performance data to generate skill learning efficiency result data characterizing the equipment's skill learning effect. Depending on different testing requirements, the skill learning efficiency result data may include data on the number of demonstrations required for learning convergence, skill generalization ability score data, or a combination of both.

[0038] S3. The task and test management module controls the intelligent mining equipment to execute a human-machine collaborative testing process based on structured interpretation data. By collecting and processing human-machine interaction-related data, test result data is generated to characterize the collaborative teaching effect of the equipment. The human-machine collaborative testing process based on structured interpretation data includes at least the following steps: S31: During the autonomous or assisted operation of the intelligent mining equipment, the interpretation and interaction module generates structured interpretation data corresponding to key decision behaviors based on the internal decision state data of the equipment control model, and outputs it through the human-machine interaction interface; S32: Under the condition of outputting the structured interpretation data, the model processing module controls a novice operator to perform collaborative operation with the intelligent mining equipment, or the novice operator takes over the operation process at a preset operation node, and the data acquisition module synchronously collects the operation performance data and human-machine interaction event data during the collaborative operation process; S33: The data acquisition module collects the physiological state data of the novice operator during the collaborative operation process, and collects the subjective feedback information filled in by the operator after the test, and converts the subjective feedback information into numerical trust parameter data. S34: The evaluation and reporting module performs data processing and index calculation operations based on the operational performance data, human-machine interaction event data, physiological state data, and numerical trust parameter data to generate collaborative teaching capability result data that characterizes the collaborative teaching effect of intelligent mining equipment. The collaborative teaching capability result data may include operational performance improvement rate data, human-machine conflict reduction rate data, human-machine interaction subjective feedback change value data, or a combination thereof.

[0039] S4. The assessment and reporting module generates a multi-level, quantitative comprehensive assessment result data report for storage and output based on the skill learning efficiency result data and collaborative teaching ability result data.

[0040] Furthermore, the number of demonstrations required for the learning to converge is determined through the following process: Step 1: Setting Expert Performance Baseline Thresholds Before training, establish a reference threshold for expert group performance used to determine convergence. M qualified expert operators (M≥3) independently complete the same target task K times (K≥3) in a simulated experimental environment, recording the core performance data (task completion time, resource consumption, job quality score, etc.) for each task. For each performance dimension, calculate the average μ and standard deviation σ of all experts' data across all tasks. For performance indicators where smaller values ​​indicate better performance, the expert performance baseline threshold is set to μ-ασ; for performance indicators where larger values ​​indicate better performance, the expert performance baseline threshold is set to μ+ασ. Here, μ is the average value of the expert group on the corresponding performance indicator, σ is the standard deviation, and α is a preset tolerance coefficient, where α is a positive real number, preferably ranging from 0.5 to 1.5.

[0041] Step 2: Execute an iterative "demonstration-learning-testing" loop. This loop incrementally uses expert demonstration data until the convergence condition is met. Initialize the cumulative number of demonstrations N = 1. Iterative process: a. Training: Use the first N expert demonstration data to train or fine-tune the parameters of the intelligent mining equipment control model. b. Testing: Allow the trained model to perform P autonomous operations (P≥5) in a standard test scenario, recording performance data. c. Evaluation and judgment: Calculate the average value of the model's core performance data from the P tests and compare it item by item with the "expert performance baseline threshold". Convergence condition: The average value of all core performance indicators of the model reaches or exceeds the threshold, indicating that learning has converged.

[0042] Non-convergence handling: If any index fails to reach the threshold, let N = N + 1, and return to step a to continue training and testing.

[0043] Step 3: Output Indicators When the convergence condition is met, the current cumulative number of demonstrations N is the "number of demonstrations required for learning convergence". This value quantifies the efficiency of the equipment control model in utilizing demonstration data for a specific task. The smaller the value of N, the stronger the imitation learning ability and the higher the efficiency. Furthermore, the tolerance coefficient α is a preset positive number used to adjust the stringency of the evaluation criteria. Its value has a clear statistical meaning: the larger the α value, the closer the performance of the equipment model is to the high-end level of the expert group, and the stricter the evaluation criteria; the smaller the α value, the more lenient the evaluation criteria. Based on statistical experience in engineering testing and the requirements for evaluation stability, the typical range of α is between 0.5 and 1.5. For example, in a preferred embodiment, α = 1 is set, meaning that the performance of the equipment model is required to stably reach or exceed the expert average level by more than one standard deviation σ.

[0044] Furthermore, the skill generalization ability score is determined through the following process: Step 1: Define a set of generalization test scenarios. In the simulated experimental environment, construct one or more "generalization test scenarios" that belong to the same task type as the standard scenarios used in the training demonstration, but with pre-defined differences. These differences aim to test the transferability and robustness of skills and typically include: Parametric perturbation-type generalization: Changing key physical or geometric parameters of the task. For example, in the "autonomous cutting" task, the Protodyakonov hardness coefficient of the coal seam is adjusted from f2 in the standard scenario to f3; the design slope of the roadway is adjusted from 5° to 8°.

[0045] Environmental interference-based generalization: Introducing external interference that did not occur or occurred at a low frequency during training. For example, in the "hydraulic support following" task, simulating a sudden, small-scale pressure on the roof; or in the "inspection" task, increasing the simulated dust concentration to test the adaptability of the vision system.

[0046] Step 2: Establish performance evaluation benchmarks For the g-th generalization test scenario, the performance benchmark value determined by experts or theory is denoted as B. g The average performance value obtained from multiple tests of the equipment model in this scenario is denoted as P. g B g It can be determined in one of the following two ways: Expert benchmarking method: Invite experts to complete a limited number of tasks (e.g., 3 times) in the generalization scenario, and take their average performance value as the benchmark B for that scenario. g .

[0047] Theoretical / Ideal Benchmark Method: If expert practice is not feasible or too costly, then based on the task objectives, a theoretically optimal or acceptable minimum performance value is set as benchmark B.g (For example, in obstacle avoidance tasks, the ideal benchmark is set as "0 collisions").

[0048] Step 3: Perform generalization testing and data collection. Using the equipment model that has reached convergence in the standard scenario (i.e., the model that has completed the "demonstration-learning" evaluation cycle), perform Q autonomous tasks (Q≥5) in each generalization test scenario. Record the core performance data of each task (consistent with the indicators used for learning efficiency evaluation, such as task completion time, accuracy, success rate, etc.), and calculate its average performance value P in that scenario. g .

[0049] Step 4: Calculate the score for each generalized test scenario g, and calculate the performance retention rate R of the equipment model. g : When the performance index is a value where a smaller value indicates better performance, the performance retention rate R... g Calculate as follows: According to R g The numerical value is converted into a "skill generalization ability score S" according to a preset scoring mapping rule. g The scoring mapping rules are stored as parameterized configurations in the system and are automatically invoked and executed by the processor in the evaluation calculation module. For example, a typical five-level scoring rule could be: If R g ≥95%, then S g =5 points (Excellent, almost no performance loss) If 85% ≤ R g <95%, then S g =4 points (Good, minimal performance loss) If 75% ≤ R g <85%, then S g =3 points (Pass, performance acceptable) If 60% ≤ R g <75%, then S g =2 points (Poor, performance deteriorates significantly) If R g <60%, then S g =1 point (poor, skill failed to generalize effectively) Step 5: Generate a comprehensive score. If multiple (G) generalization test scenarios are defined, the final "skill generalization ability score S" will be generated. final It can be generated in any of the following ways: Average rating: Worst rating (more stringent evaluation): S final =min(S1,S2,…,S) G The overall score is S. final This is the final quantitative indicator; the higher the score, the stronger the robustness and adaptability of the acquired skills.

[0050] Furthermore, the operational performance improvement rate is a comprehensive evaluation index used to characterize the effect of equipment interpretation assistance on improving operator performance. It includes at least two sub-indicators: task efficiency improvement rate and operational accuracy improvement rate. Specifically, in the absence of interpretation assistance, the core performance data of a novice completing a standard task is recorded: task completion time T. base Operational accuracy A base Energy consumption E base In the explained and assisted mode, record the performance data for completing the same or equivalent tasks: task completion time T. assist Operational accuracy A assist Energy consumption E assist .

[0051] The improvement rate is calculated based on a preset formula; the task efficiency improvement rate is... Improvement rate of work accuracy A positive improvement rate and a larger value indicate that the equipment's explanatory assistance has a more significant effect on improving operator performance.

[0052] In one implementation, the task efficiency improvement rate R can be... T With the improvement rate of work accuracy R A Together, they serve as a quantitative result of the operational performance improvement rate; in another embodiment, R can be weighted according to preset weights. T With R A Weighted summation is performed to obtain a composite value for the single operational performance improvement rate. Furthermore, the human-machine conflict reduction rate: This sub-index measures the effectiveness of equipment interpretation behavior in reducing unintended human-machine interaction conflicts. Specifically: Define and count "human-machine conflict events" occurring during testing, including but not limited to: operator performing emergency manual overwrite, system forcibly interrupting automatic processes due to safety rules being triggered, and operator actions and equipment automatic actions being logically mutually exclusive (e.g., adjusting in opposite directions simultaneously). Count the total number C of conflict events occurring in the unexplained assistance mode. base And the total number of conflict events C in the explained auxiliary mode. assist Calculate the rate of decrease in human-machine conflict. The higher the value, the better the equipment's interpretive capabilities can promote human-machine understanding and reduce unintended conflicts.

[0053] Furthermore, the subjective feedback change value of human-computer interaction: This sub-index is used to characterize the changes in the operator's subjective feedback on the equipment interaction behavior under different test conditions.

[0054] After each test (unassisted baseline test and assisted collaborative test), the data acquisition module guides the operator to complete a pre-validated human-machine interaction subjective feedback scale. This scale includes multiple quantitative items reflecting the operator's willingness to adopt recommendations regarding the consistency and predictability of equipment decision outputs. Each item uses a Likert multi-level scoring method to obtain numerical feedback data. The average feedback score S under the unassisted baseline test condition is calculated separately. base And the average feedback score S under conditions of assisted collaborative testing. assist And calculate the feedback change value ΔS = S assist -S base The feedback change value is used as a numerical human-computer interaction feedback parameter in the statistical calculation of collaborative teaching ability results data, and is used to reflect the change range of subjective human-computer interaction feedback under different test conditions.

[0055] like Figure 2 As shown, the present invention also provides an intelligent mining equipment collaborative learning capability testing system. The system is deployed on a computer device or server and is used to control the intelligent mining equipment to execute the test process in a simulated experimental environment, and to collect, process and analyze multi-source data generated during the test to generate test result data for evaluating human-machine collaborative learning capabilities.

[0056] The system includes at least: a task and test management module, a data acquisition module, a model processing module, an interpretation and interaction module, and an evaluation and reporting module. Each module is implemented by the processor executing program instructions stored in the memory.

[0057] The task and test management module is used to construct a composite mining task model in a simulated experimental environment and to uniformly schedule and manage each functional module according to a preset test process. The composite mining task model includes environmental uncertainty parameters, autonomous decision-making constraints, and task objective parameters. The task and test management module is communicatively connected to the model processing module, data acquisition module, interpretation and interaction module, and evaluation and reporting module, respectively, and is used to issue task configuration parameters and test control instructions.

[0058] The data acquisition module is communicatively connected to the model processing module, the interpretation and interaction module, and the evaluation and reporting module. It is used to collect and record multi-source test data during the testing process. The multi-source test data includes at least the operation instruction sequence data of human experts or novice operators, environmental state data of the simulated experimental environment, task execution performance data of intelligent mining equipment, human-computer interaction event data, physiological state related data, and subjective feedback information data. The multi-source test data is stored in the memory.

[0059] The model processing module is communicatively connected to the task and test management module and the data acquisition module. It is used to call the multimodal demonstration data stored in the memory, perform imitation learning training or parameter fine-tuning operations on the control model of the intelligent mining equipment, and control the updated equipment control model to perform composite mining tasks in preset standard test scenarios or generalized test scenarios.

[0060] The interpretation and interaction module is communicatively connected to the model processing module and the data acquisition module. It is used to generate structured interpretation data corresponding to key decision behaviors based on the internal decision state data of the intelligent mining equipment control model, and output the structured interpretation data through the human-computer interaction interface.

[0061] The assessment and reporting module is connected to the data acquisition module and the task and test management module. It is used to perform index calculations based on the collected data, generate skill learning efficiency result data and collaborative teaching ability result data, and form a multi-level, quantitative comprehensive assessment result data report.

[0062] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for testing the collaborative learning ability of intelligent mining equipment, characterized in that, The testing method includes the following steps: S1. Construct a composite mining task model with environmental uncertainty parameters and autonomous decision-making constraints in a simulated experimental environment, and initialize the control model of the intelligent mining equipment to be tested. S2. Collect multimodal demonstration data of human expert operators completing complex mining tasks, conduct imitation learning training or parameter fine-tuning of the control model, and obtain equipment skill learning efficiency results data through an incremental demonstration-learning-testing process, including: S21. Collect multimodal demonstration data, including operation command sequences, environmental conditions, and data related to expert attention focus. S22. Call the multimodal demonstration data to train or fine-tune the control model and generate an updated control model. S23. Use the updated control model to execute the task in a preset standard test scenario and collect performance data, including task completion time, resource consumption, and job quality parameters. S24. Generate skill learning efficiency results data based on performance data, including the number of demonstrations required for learning convergence, skill generalization ability score, or a combination of both. S3. The intelligent mining equipment will execute a human-machine collaborative testing process based on structured interpreted data. By collecting and processing human-machine interaction data, test result data will be generated to characterize the collaborative teaching effectiveness of the equipment, including: S31. During equipment operation, the internal decision-making state data based on the equipment control model generates and outputs structured explanatory data corresponding to key decision-making behaviors. S32. After outputting the interpretation data, allow novice operators to work collaboratively with the equipment or take over the operation, and collect operational performance and human-machine interaction event data. S33. Collect physiological state data and subjective feedback information of novice operators, and convert the subjective feedback information into numerical trust parameter data. S34. Generate collaborative teaching ability result data based on the above data, including operational performance improvement rate, human-computer conflict reduction rate, human-computer interaction subjective feedback change value or a combination thereof. S4. Based on the skill learning efficiency results data and the multi-source data collected during the collaborative testing process, perform fusion processing and index calculation to generate a multi-level, quantitative evaluation result of the human-machine collaborative learning capability of intelligent mining equipment.

2. The intelligent calculation method for stacking and digging points of an unmanned loader according to claim 1, characterized in that, The number of demonstrations required for learning convergence is determined through the following process: In a simulated experimental environment, based on the core performance data of multiple expert operators completing the same target task, the average value μ and standard deviation σ of the expert group on each performance index are calculated, and combined with a preset tolerance coefficient α, the expert performance baseline threshold for judging learning convergence is determined; In the process of incrementally introducing expert demonstration data, the intelligent mining equipment control model is repeatedly subjected to demonstration learning training and autonomous operation testing under standard test scenarios, and the model performance data obtained from the test is compared with the expert performance baseline threshold. When the average performance of the model on each core performance index reaches or exceeds the corresponding threshold, the model learning convergence is determined, and the cumulative number of expert demonstrations used at this time is determined as the number of demonstrations required for learning convergence.

3. The intelligent calculation method for stacking and digging points of an unmanned loader according to claim 2, characterized in that, The tolerance coefficient α is a preset positive number used to adjust the strictness of the evaluation criteria.

4. The method according to claim 1, characterized in that, The skill generalization ability score is determined as follows: at least one generalization test scenario of the same task type as the training demonstration standard scenario is constructed in the simulated experimental environment, but with preset differences. The intelligent mining equipment control model, which has been learned and converged in the standard scenario, performs autonomous operations multiple times to obtain the average performance value of the model. The performance evaluation benchmark value is determined based on the expert operation results or the preset theoretical performance standard, and the corresponding skill generalization ability score is generated based on the comparison result between the model average performance value and the benchmark value. When there are multiple generalization test scenarios, the skill generalization ability scores of each scenario are merged to generate the final comprehensive skill generalization ability score.

5. The method according to claim 1, characterized in that, The operational performance improvement rate is determined by comparing the performance data of novice operators completing the same or equivalent task in unassisted mode and assisted mode. The performance data includes at least the task completion time and the accuracy of the operation. Based on the performance data, the task efficiency improvement rate and the operation accuracy improvement rate are calculated respectively, and the operation performance improvement rate is a combination result or a weighted composite result of the two.

6. The method according to claim 1, characterized in that, The reduction rate of human-machine conflict is determined by statistically analyzing the number of human-machine conflict events that occur during the execution of tasks by intelligent mining equipment in both unassisted and assisted modes. These human-machine conflict events include at least emergency manual overwriting, forced interruption of automatic processes, and mutually exclusive operation commands. The reduction rate of human-machine conflict is calculated based on the total number of conflict events in unassisted mode and the total number of conflict events in assisted mode.

7. The method according to claim 1, characterized in that, The change value of the human-computer interaction subjective feedback is determined by comparing the quantitative subjective feedback data obtained by the operator after the unassisted test and the assisted test, and the change value of the feedback is used as one of the quantitative parameters for the assessment of collaborative teaching ability.

8. A collaborative learning capability testing system for intelligent mining equipment, characterized in that, include: The task and test management module is used to construct a composite mining task model and schedule the test process; The data acquisition module collects and stores operation commands, environmental status, task execution performance, human-computer interaction events, physiological status, and subjective feedback data. The model processing module performs imitation learning training or parameter fine-tuning on the intelligent mining equipment control model based on multimodal demonstration data, controlling the updated model to execute composite mining tasks. The interpretation and interaction module generates and outputs structured interactive information based on the internal decision-making state data of the control model. The evaluation and reporting module performs fusion processing and index calculation on multi-source data to generate skill learning efficiency, collaborative teaching ability, and comprehensive evaluation results. All modules are connected via a communication interface and operate collaboratively under the unified scheduling of the processor.

Images