Automatic scoring methods, devices, and storage media for students' robotic arm operation tasks.

Abstract

This invention relates to the field of artificial intelligence technology and discloses an automatic scoring method, device, and storage medium for students' robotic arm operation tasks. The method includes: acquiring multimodal monitoring information for the robotic arm operation task; preprocessing the multimodal monitoring information to obtain preprocessed information; acquiring preset scoring rules; acquiring a trajectory recognition model based on the multimodal data; analyzing the preprocessed information based on the trajectory recognition model to generate a trajectory recognition result; and scoring based on the preset scoring rules and the trajectory recognition result to generate a scoring result. By improving traditional scoring methods, this invention enhances fairness and scoring efficiency by employing automatic machine recognition for analysis and scoring, and improves scoring accuracy by using multi-dimensional monitoring data.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically to an automatic scoring method for students' robotic arm operation tasks, an automatic scoring device for students' robotic arm operation tasks, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of information technology, the education sector is increasingly inclined to adopt online education models to provide a more flexible and convenient way of learning. However, traditional hands-on courses, especially those involving complex skills such as robotic arm operation, still face challenges in regulation and assessment.

[0003] In traditional hands-on courses, students learn and master specific skills and operating procedures by operating robotic arms themselves. This practical training not only helps students understand the challenges and requirements of real-world work environments but also cultivates their hand-eye coordination and problem-solving skills.

[0004] However, robotic arm operation involves multiple skill levels, including programming, operation, and debugging. Traditional assessment methods often rely on manual monitoring by teachers, which carries the risks of blind spots in supervision and unfair evaluation. Therefore, a method that can automatically monitor and evaluate student operations is needed to improve the fairness and efficiency of assessment. Summary of the Invention

[0005] To overcome the aforementioned technical problems in the prior art, embodiments of the present invention provide an automatic scoring method, device, and storage medium for students' robotic arm operation tasks. By improving the traditional scoring method, on the one hand, it adopts machine automatic recognition for analysis and scoring, thereby improving fairness and scoring efficiency; on the other hand, it uses multi-dimensional monitoring data for scoring, thereby improving scoring accuracy.

[0006] To achieve the above objectives, embodiments of the present invention provide an automatic scoring method for students' robotic arm operation tasks. The method includes: acquiring multimodal monitoring information for the robotic arm operation task; preprocessing the multimodal monitoring information to obtain preprocessed information; acquiring preset scoring rules; acquiring a trajectory recognition model based on the multimodal data; analyzing the preprocessed information based on the trajectory recognition model to generate a trajectory recognition result; and scoring based on the preset scoring rules and the trajectory recognition result to generate a scoring result.

[0007] Preferably, the multimodal monitoring information includes movement trajectory information, image monitoring information, sound monitoring information, and text information. The preprocessing of the multimodal monitoring information to obtain preprocessed information includes: performing trajectory optimization processing on the movement trajectory information to obtain a processed trajectory; performing enhancement and correction processing on the image monitoring information to obtain a processed image; performing sound denoising processing on the sound monitoring information to obtain processed sound; performing text denoising, text cleaning, word segmentation, and part-of-speech tagging processing on the text information sequentially to obtain processed text; and generating preprocessed information based on the processed trajectory, the processed image, the processed sound, and the processed text.

[0008] Preferably, the step of performing trajectory optimization processing on the movement trajectory information to obtain the processed trajectory includes: extracting multiple trajectory data points of the robotic arm based on the movement trajectory information; performing a linear fitting operation on all trajectory data points to generate a fitted trajectory; obtaining the projection distance between each trajectory data point and the fitted trajectory; determining abnormal distances where the projection distance is greater than a preset distance threshold; and smoothing the corresponding trajectory data points based on the abnormal distances to obtain the processed trajectory.

[0009] Preferably, the method further includes: acquiring an initial GAN ​​network before acquiring the trajectory recognition model based on multimodal data; acquiring historical trajectory data and corresponding historical multimodal data of the robotic arm operation; adjusting the weights of each data type in the historical multimodal data based on a preset representation learning model to generate adjusted weights; generating a training dataset based on the historical trajectory data, the historical multimodal data, and the adjusted weights; and training the initial GAN ​​network based on the training dataset to generate a trajectory recognition model based on multimodal data.

[0010] Preferably, the step of adjusting the weights of each data point in the historical multimodal data based on a preset representation learning model to generate adjusted weights includes: performing importance analysis on each data point in the historical multimodal data based on the preset representation learning model to generate the most important data, the second most important data, and the remaining data; determining the first weight of the most important data, the second weight of the second most important data, and the third weight of the remaining data; and generating adjusted weights based on the first weight, the second weight, and the third weight.

[0011] Preferably, the step of analyzing the preprocessed information based on the trajectory recognition model to generate a trajectory recognition result includes: performing multimodal feature extraction on the preprocessed information to obtain multimodal data features, wherein the multimodal data features are related to the movement trajectory of the robotic arm; performing feature fusion on the multimodal data features to obtain fused features; and performing multimodal analysis on the fused features based on the trajectory recognition model to generate a trajectory recognition result.

[0012] Preferably, the preset scoring rules include program scoring rules, job result scoring rules, and job process scoring rules. The step of scoring based on the preset scoring rules and the trajectory recognition results to generate a scoring result includes: acquiring the job program corresponding to the robotic arm operation; analyzing the job program based on the program scoring rules to generate a first score; analyzing the trajectory recognition results based on the job process scoring rules to generate a second score; acquiring monitoring information of the completed robotic arm operation; analyzing the monitoring information based on the job result scoring rules to generate a third score; and generating a scoring result based on the first score, the second score, and the third score.

[0013] Preferably, the method further includes: when the trajectory recognition result indicates the existence of a trajectory anomaly, determining the anomaly category of the trajectory recognition result based on the processed information; generating early warning information based on the anomaly category; obtaining a preset reference answer based on the trajectory recognition result and the anomaly category; and outputting the early warning information and the preset reference answer.

[0014] On the other hand, the present invention also provides an automatic scoring device for students' robotic arm operation tasks. The device includes: an information acquisition unit for acquiring multimodal monitoring information for robotic arm operation tasks; a preprocessing unit for preprocessing the multimodal monitoring information to obtain preprocessed information; a rule acquisition unit for acquiring preset scoring rules; a trajectory recognition unit for acquiring a trajectory recognition model based on multimodal data, analyzing the preprocessed information based on the trajectory recognition model, and generating a trajectory recognition result; and a scoring unit for scoring based on the preset scoring rules and the trajectory recognition result to generate a scoring result.

[0015] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided by the present invention.

[0016] The present invention has at least the following technical effects through the technical solution provided by the present invention:

[0017] By employing computer-based recognition to analyze and score students' robotic arm operation tasks, a more objective and fair scoring method can be used, improving fairness. At the same time, it eliminates the need for teachers to manually observe each operation task in real time. Instead, multi-modal data is used to conduct a comprehensive multi-dimensional scoring of the operation tasks, improving scoring efficiency, reducing subjective influence, and improving scoring accuracy.

[0018] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0020] Figure 1 This is a flowchart illustrating the specific implementation of the automatic scoring method for students' robotic arm operation tasks provided in this embodiment of the invention.

[0021] Figure 2 This is a flowchart illustrating the specific implementation of preprocessing multimodal monitoring information to obtain preprocessed information, as provided in this embodiment of the invention.

[0022] Figure 3 This is a schematic diagram of a trajectory recognition model based on multimodal data provided in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of the structure of an automatic scoring device for students' robotic arm operation tasks provided in an embodiment of the present invention. Detailed Implementation

[0024] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0025] In this invention, the terms "system" and "network" are used interchangeably. "Multiple" refers to two or more; therefore, in this invention, "multiple" can also be understood as "at least two." "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, it should be understood that in the description of this invention, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0026] Please see Figure 1 This invention provides an automatic scoring method for students' robotic arm operation tasks, the method comprising:

[0027] S10) Obtain multimodal monitoring information for robotic arm operation;

[0028] S20) Preprocess the multimodal monitoring information to obtain preprocessed information;

[0029] S30) Obtain preset scoring rules;

[0030] S40) Obtain a trajectory recognition model based on multimodal data, analyze the preprocessed information based on the trajectory recognition model, and generate trajectory recognition results;

[0031] S50) Based on the preset scoring rules and the trajectory recognition results, a score is generated.

[0032] In one possible implementation, firstly, regarding the multimodal monitoring information of the robotic arm operation, in this embodiment, multimodal monitoring information during the operation can be collected by a camera set in the operation scene of the robotic arm. This multimodal monitoring information includes, but is not limited to, images, sounds, and other messages during the operation. At the same time, other data is also obtained through API interfaces, such as by connecting to the school's education system API interface to obtain students' student IDs, names, real-time problem-solving data, as well as the robotic arm's alarm logs and program code, all of which are used as the aforementioned multimodal monitoring information.

[0033] After obtaining the above multimodal monitoring information, the original information may have certain deviations due to interference and other reasons. Therefore, in order to improve the accuracy of subsequent analysis and scoring, it is necessary to perform preprocessing operations to obtain more accurate multimodal monitoring information.

[0034] Please see Figure 2In this embodiment of the invention, the multimodal monitoring information includes movement trajectory information, image monitoring information, sound monitoring information, and text information. The preprocessing of the multimodal monitoring information to obtain preprocessed information includes:

[0035] S21) Perform trajectory optimization processing on the movement trajectory information to obtain the processed trajectory;

[0036] S22) Perform enhancement and correction processing on the image monitoring information to obtain a processed image;

[0037] S23) Perform sound denoising processing on the sound monitoring information to obtain the processed sound;

[0038] S24) Perform text denoising, text cleaning, word segmentation and part-of-speech tagging on the text information in sequence to obtain the processed text;

[0039] S25) Generate preprocessed information based on the processed trajectory, the processed image, the processed sound, and the processed text.

[0040] In one possible implementation, the movement trajectory information in the multimodal information is first processed for trajectory optimization. For example, in this embodiment, before the student controls the robotic arm to perform the operation, the student inputs the control program he wrote on the designated platform of the education system. The platform writes the control program to the designated robotic arm main control unit and then controls the robotic arm to perform the corresponding operation.

[0041] In one specific embodiment, the operation involves controlling a robotic arm to move a pre-set pallet to a designated position. During the movement, the robotic arm moves along a corresponding trajectory according to the control program. In the process of collecting multimodal monitoring information, the movement trajectory information is used as one of the multimodal monitoring information. For example, the movement point data of the robotic arm can be read through the aforementioned API to form its movement trajectory information, or the movement point data of the robotic arm can be generated by a camera based on real-time monitoring footage to form the aforementioned movement trajectory information.

[0042] In this embodiment of the invention, the step of performing trajectory optimization processing on the movement trajectory information to obtain a processed trajectory includes: extracting multiple trajectory data points of the robotic arm based on the movement trajectory information; performing a linear fitting operation on all trajectory data points to generate a fitted trajectory; obtaining the projection distance between each trajectory data point and the fitted trajectory; determining abnormal distances where the projection distance is greater than a preset distance threshold; and smoothing the corresponding trajectory data points based on the abnormal distances to obtain the processed trajectory.

[0043] Specifically, multiple trajectory data points of the robotic arm are extracted based on the aforementioned movement trajectory information. To determine which abnormal and noise points exist among these trajectory data points, a linear fitting operation is performed on all trajectory data points to generate a fitted trajectory. Then, the projection distance between each trajectory data point and the fitted trajectory is further obtained, i.e., the distance between each trajectory data point and the actual running trajectory of the robotic arm is determined. Next, abnormal distances greater than a preset distance threshold are obtained from the aforementioned projection distances. For example, in this embodiment, the step size between every two adjacent trajectory data points is 1mm, so the preset distance threshold is set to 3mm. When the distance between a point and the fitted trajectory exceeds 3mm, that point can be determined as an abnormal trajectory point. At this point, the trajectory point is smoothed, for example, by restoring it to its projection point on the fitted trajectory. This achieves optimization of the original trajectory data points and obtains the processed trajectory.

[0044] In practical applications, when using an optical camera to capture the movement trajectory of a robotic arm, some anomalies may occur, causing the formed movement trajectory to deviate from the actual trajectory. Normally, the robotic arm's movement trajectory should match the trajectory set in its program. Therefore, optimization and verification processing can be performed. In another embodiment, after obtaining the aforementioned multiple trajectory data points, the student's control program is further acquired, and the desired movement trajectory of the robotic arm is determined based on the control program. At this point, it is determined whether the distance between the aforementioned multiple trajectory data points and the desired movement trajectory exceeds a preset value. If so, the point is determined to be an abnormal trajectory point, and smoothing processing is then performed to obtain the processed trajectory.

[0045] At this point, further enhancement and correction processing is performed on the image monitoring information, including but not limited to scaling, cropping, and color correction, to obtain a processed image. Through the above processing, the image clarity and contrast are improved, and the tilt angle is adjusted to obtain an image more conducive to subsequent processing. Sound monitoring information is then subjected to sound denoising processing. Specifically, existing sound denoising algorithms can be used to remove background noise from the sound monitoring information, while simultaneously enhancing the sound information during the robotic arm's operation to obtain processed sound. This processed sound includes, but is not limited to, footsteps, conversations, and collision sounds. Furthermore, text information is sequentially subjected to text denoising, text cleaning, word segmentation, and part-of-speech tagging. For example, in this embodiment, the text information includes, but is not limited to, students' program source code and operation log information. By performing the above processing, it is transformed into a data format suitable for computer recognition and analysis. Finally, the processed information is used to generate preprocessed information.

[0046] In this embodiment of the invention, by introducing multimodal data on the basis of simple image recognition, such as conversations during robotic arm operation, log information during program execution, and limit indicator lights triggered during robotic arm movements, the performance of students' robotic arm operation can be evaluated from multiple dimensions, providing a sufficient and comprehensive data foundation for subsequent accurate scoring.

[0047] At this point, a trajectory recognition model based on multimodal data is obtained, and the preprocessed information is analyzed to generate the corresponding trajectory recognition results. Before this, a trajectory recognition model based on multimodal data needs to be created.

[0048] In this embodiment of the invention, the method further includes: acquiring an initial GAN ​​network before acquiring the trajectory recognition model based on multimodal data; acquiring historical trajectory data and corresponding historical multimodal data of the robotic arm operation; adjusting the weights of each data type in the historical multimodal data based on a preset representation learning model to generate adjusted weights; generating a training dataset based on the historical trajectory data, the historical multimodal data, and the adjusted weights; and training the initial GAN ​​network based on the training dataset to generate a trajectory recognition model based on multimodal data.

[0049] In one possible implementation, an initial GAN ​​network is first obtained, followed by historical trajectory data and corresponding historical multimodal data of the robotic arm operation. In practical applications, since the multimodal data contains multi-dimensional data, but not every dimension of data has the same importance, it is necessary to assign weights to it to focus more on the most important data, while other data are used to assist it, so as to comprehensively improve the recognition accuracy of the robotic arm operation.

[0050] In this embodiment of the invention, adjusting the weights of each data point in the historical multimodal data based on a preset representation learning model to generate adjusted weights includes: performing importance analysis on each data point in the historical multimodal data based on the preset representation learning model to generate the most important data, the second most important data, and the remaining data; determining the first weight of the most important data, the second weight of the second most important data, and the third weight of the remaining data; and generating adjusted weights based on the first weight, the second weight, and the third weight.

[0051] In one possible implementation, importance analysis is first performed on each data point in the historical multimodal data according to a preset representation learning model. This preset representation learning model can be, for example, an existing representation learning model or a joint learning model. This representation learning model or joint learning model can integrate and perform weight analysis on each data point in the multimodal data, and determine which features are most critical to the recognition results based on the importance of different features. This can more efficiently integrate the data from various modalities, enabling the model to comprehensively consider information from multiple dimensions when making judgments or predictions. This reduces misjudgments or uncertainties caused by insufficient or missing information from a single modality, improves overall processing efficiency, and avoids misjudgments caused by insufficient data from a single data source.

[0052] Specifically, the aforementioned multimodal data is divided into most important data, second most important data, and remaining data. Of course, those skilled in the art can further refine the classification of data importance levels according to actual needs, and all such classifications should fall within the protection scope of this invention. Further details will not be elaborated here. After analyzing the historical multimodal data using a common representation learning model, the first weight of the most important data, the second weight of the second most important data, and the third weight of the remaining data are determined.

[0053] After redefining the weights for each modality of data, a training dataset is generated based on the historical trajectory data, historical multimodal data, and adjusted weights. This dataset is then input into the initial GAN ​​network for training to generate a trajectory recognition model based on multimodal data. For example, please refer to [link to relevant documentation]. Figure 3 This is a schematic diagram of a trajectory recognition model based on multimodal data provided in an embodiment of the present invention. The preprocessed information is then analyzed using the aforementioned model to generate trajectory recognition results.

[0054] In this embodiment of the invention, the step of analyzing the preprocessed information based on the trajectory recognition model to generate a trajectory recognition result includes: performing multimodal feature extraction on the preprocessed information to obtain multimodal data features, wherein the multimodal data features are related to the movement trajectory of the robotic arm; performing feature fusion on the multimodal data features to obtain fused features; and performing multimodal analysis on the fused features based on the trajectory recognition model to generate a trajectory recognition result.

[0055] In one possible implementation, students program a robotic arm to move a colored block from its original position to a target position. During this process, the movement of the robotic arm's end effector must be within a predetermined motion space, such as a semicircle with a radius of 320. During monitoring, multiple trajectory lines of the robotic arm's operation are recorded and analyzed using the aforementioned trajectory recognition model. For example, in this embodiment, through comparative analysis, it is determined that the robotic arm's movement trajectory characteristics match the expected trajectory characteristics, and the relevant multimodal data characteristics of the robotic arm are also normal. Therefore, a trajectory recognition result indicating a normal movement trajectory is generated.

[0056] In this embodiment of the invention, the features of the program / text include, but are not limited to: number of lines of code, number of lines of comments, identifiers (such as variable names, function names, etc.), control flow structure (such as loops, conditional statements), function calls, etc.; the features of the image / video include, but are not limited to, color features, shape features, spatial relationship features, texture features, etc.

[0057] In the second embodiment, although the trajectory characteristics are determined to match the expected trajectory characteristics during the trajectory recognition process, at the start of the movement, the monitoring image of the robotic arm shows that its starting position exceeds the limit, hence the indicator light is red, thus confirming that the starting position exceeds the limit. Simultaneously, an alarm message indicating that the starting position exceeds the limit is received from the robotic arm control software via the API interface, thus generating a trajectory recognition result indicating that the starting position exceeds the limit.

[0058] In the third embodiment, when the movement coordinates of the robotic arm are obtained, the robotic arm is moved out of the set action space. At this time, on the one hand, the control software will issue an over-limit alarm; on the other hand, the indicator light on the robotic arm will change from green to red; and on the third hand, the trajectory point of the robotic arm will be located outside the set action space. Therefore, during the scoring process, after analyzing the above multimodal data through the trajectory recognition model, a trajectory recognition result of the robotic arm position exceeding the limit is generated.

[0059] In the fourth embodiment, during the scoring of the robotic arm operation, the robotic arm was monitored to emit a jamming sound / palletizing collision sound. Therefore, in the subsequent scoring process, during the multimodal data recognition of the robotic arm operation, the multimodal data features were analyzed using a common representation learning model, and the sound features were identified as the most important data features. The analysis and judgment were then performed to generate a preliminary analysis result of "robotic arm jamming / palletizing collision". In this embodiment, although the robotic arm's motion trajectory was found to be consistent with the expected trajectory during the comparison process (the robotic arm trajectory exceeding the Z-axis limit may be identified as a normal trajectory), the trajectory recognition result based on the motion trajectory was inconsistent with the above preliminary analysis result. However, based on the weight of the sound features, the preliminary analysis result was still adopted as the final trajectory recognition result to ensure the accuracy of the final scoring of the robotic arm operation.

[0060] In this embodiment of the invention, by using multimodal data to perform multidimensional comprehensive analysis on the trajectory recognition of robotic arm operations, the analysis bias caused by a single data source can be effectively solved. At the same time, by combining a representation learning model to dynamically adjust the weight of each data in the multimodal data, the most important data source is given priority to objectively evaluate the operation, thereby greatly improving the accuracy of the final analysis.

[0061] After generating the trajectory recognition result of the robotic arm operation, the operation is further scored according to a pre-set scoring rule. In this embodiment of the invention, the pre-set scoring rule includes a program scoring rule, an operation result scoring rule, and an operation process scoring rule. The scoring based on the pre-set scoring rule and the trajectory recognition result to generate a scoring result includes: obtaining the operation program corresponding to the robotic arm operation; analyzing the operation program based on the program scoring rule to generate a first score; analyzing the trajectory recognition result based on the operation process scoring rule to generate a second score; obtaining the result monitoring information of the completion of the robotic arm operation, analyzing the result monitoring information based on the operation result scoring rule to generate a third score; and generating a scoring result based on the first score, the second score, and the third score.

[0062] In one possible implementation, the teacher can pre-set grading rules, which may include procedural grading rules, assignment result grading rules, and assignment process grading rules, etc. For example, please refer to the table below for a specific example of grading rules:

[0063]

[0064] In the specific implementation process, the operation program corresponding to the robotic arm operation is first obtained. This operation program is the robotic arm control program uploaded by the student. Then, the operation program is analyzed according to the above program scoring rules to generate a first score. The second score is generated according to the operation process scoring rules, such as in this embodiment, the trajectory recognition results are analyzed based on whether the action exceeds the limit. The third score is generated according to the operation result scoring rules and the result monitoring information is analyzed.

[0065] For example, in one specific embodiment, by initially analyzing the robotic arm operation and determining that the operation is normal, all features are then compared with a dataset that has been pre-marked as "correct" or "out of bounds". It is found that there is a motion trajectory in the student's work that is marked as "out of bounds" in the dataset, that is, an error is found in the student's work. Therefore, the corresponding score correction is automatically performed, and the final score result is obtained.

[0066] In this embodiment of the invention, by quantitatively analyzing and scoring students' robotic arm operation tasks from multiple dimensions, on the one hand, the accuracy of automatic grading of students' control tasks is effectively improved, enhancing the user experience; on the other hand, it greatly reduces the workload of teachers, improves their work efficiency, reduces their work pressure, and meets practical needs.

[0067] In practical applications, after discovering problems with students' assignments, it is desirable to receive automatic and targeted prompts or answers to these problems. However, current technologies do not provide solutions to these technical issues.

[0068] In this embodiment of the invention, the method further includes: when the trajectory recognition result indicates the existence of a trajectory anomaly, determining the anomaly category of the trajectory recognition result based on the processed information; generating warning information based on the anomaly category; obtaining a preset reference answer based on the trajectory recognition result and the anomaly category; and outputting the warning information and the preset reference answer.

[0069] In one possible implementation, teachers can pre-classify, organize, and archive students' common mistakes and problems. During subsequent homework grading, when problems are detected in students' work, the system automatically performs anomaly analysis on the trajectory recognition results from multiple dimensions based on multimodal data to determine the anomaly category and generate corresponding warning information. On the other hand, the system also finds corresponding preset reference answers based on the trajectory recognition results and anomaly categories and outputs them to students so that they can improve and correct their mistakes in a timely manner, avoiding the recurrence of similar problems, thereby effectively improving students' learning efficiency and learning outcomes.

[0070] Please see Figure 4Based on the same inventive concept, this invention also provides an automated grading device for students' robotic arm operation tasks. The device includes: an information acquisition unit for acquiring multimodal monitoring information for the robotic arm operation tasks; a preprocessing unit for preprocessing the multimodal monitoring information to obtain preprocessed information; a rule acquisition unit for acquiring preset scoring rules; a trajectory recognition unit for acquiring a trajectory recognition model based on the multimodal data, analyzing the preprocessed information based on the trajectory recognition model, and generating a trajectory recognition result; and a scoring unit for scoring based on the preset scoring rules and the trajectory recognition result to generate a scoring result.

[0071] Furthermore, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the present invention.

[0072] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention.

[0073] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not describe the various possible combinations separately.

[0074] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0075] Furthermore, various different implementations of the present invention can be combined arbitrarily, as long as they do not violate the spirit of the present invention, they should also be regarded as the content disclosed in the present invention.

Claims

1. An automatic scoring method for students' robotic arm operation tasks, characterized in that, The method includes: Acquire multimodal monitoring information for robotic arm operations, including movement trajectory information, image monitoring information, sound monitoring information, and text information; The multimodal monitoring information is preprocessed to obtain preprocessed information; Obtain preset scoring rules, which include program scoring rules, job result scoring rules, and job process scoring rules; A trajectory recognition model based on multimodal data is obtained, and the preprocessed information is analyzed based on the trajectory recognition model to generate trajectory recognition results; A score is generated based on the preset scoring rules and the trajectory recognition results; The preprocessing of the multimodal monitoring information to obtain preprocessed information includes: Perform trajectory optimization processing on the movement trajectory information to obtain the processed trajectory; The image monitoring information is subjected to enhancement and correction processing to obtain a processed image; Perform noise reduction processing on the sound monitoring information to obtain the processed sound; The text information is sequentially subjected to text denoising, text cleaning, word segmentation and part-of-speech tagging to obtain the processed text; Preprocessed information is generated based on the processed trajectory, the processed image, the processed sound, and the processed text; The step of analyzing the preprocessed information based on the trajectory recognition model to generate trajectory recognition results includes: Multimodal feature extraction is performed on the preprocessed information to obtain multimodal data features, which are related to the movement trajectory of the robotic arm; The multimodal data features are fused to obtain fused features; Based on the trajectory recognition model, multimodal analysis is performed on the fused features to generate trajectory recognition results; The step of scoring based on the preset scoring rules and the trajectory recognition results to generate a scoring result includes: Obtain the operation program corresponding to the robotic arm operation; The job program is analyzed based on the program scoring rules to generate a first score; The trajectory recognition results are analyzed based on the work process scoring rules to generate a second score; The system acquires monitoring information on the completion of the robotic arm operation, analyzes the monitoring information based on the operation result scoring rules, and generates a third score. A scoring result is generated based on the first score, the second score, and the third score; The method further includes: If the trajectory recognition result indicates the existence of a trajectory anomaly, the anomaly category of the trajectory recognition result is determined based on the processed information; Early warning information is generated based on the anomaly category; Based on the trajectory recognition results and the anomaly category, a preset reference answer is obtained; Output the warning information and the preset reference answer.

2. The method according to claim 1, characterized in that, The step of performing trajectory optimization processing on the movement trajectory information to obtain the processed trajectory includes: Based on the movement trajectory information, multiple trajectory data points of the robotic arm are extracted; Perform a linear fitting operation on all trajectory data points to generate a fitted trajectory; Obtain the projected distance between each trajectory data point and the fitted trajectory; Identify abnormal distances where the projected distance exceeds a preset distance threshold; Based on the abnormal distance, the corresponding trajectory data points are smoothed to obtain the processed trajectory.

3. The method according to claim 1, characterized in that, The method further includes: Before acquiring the trajectory recognition model based on multimodal data, an initial GAN ​​network is acquired; Acquire the historical trajectory data and corresponding historical multimodal data of the robotic arm's operation; The weights of each data point in the historical multimodal data are adjusted based on a preset representation learning model to generate adjusted weights. A training dataset is generated based on the historical trajectory data, the historical multimodal data, and the adjusted weights; The initial GAN ​​network is trained based on the training dataset to generate a trajectory recognition model based on multimodal data.

4. The method according to claim 3, characterized in that, The step of adjusting the weights of each data point in the historical multimodal data based on a preset representation learning model to generate adjusted weights includes: Based on the preset representation learning model, importance analysis is performed on each type of data in the historical multimodal data to generate the most important data, the second most important data, and the remaining data; Determine the first weight of the most important data, the second weight of the second most important data, and the third weight of the remaining data; Adjusted weights are generated based on the first weight, the second weight, and the third weight.

5. An automatic scoring device for students' robotic arm operation tasks, characterized in that, The device includes: The information acquisition unit is used to acquire multimodal monitoring information for the operation of the robotic arm, including movement trajectory information, image monitoring information, sound monitoring information and text information; The preprocessing unit is used to preprocess the multimodal monitoring information to obtain preprocessed information; The rule acquisition unit is used to acquire preset scoring rules, which include program scoring rules, job result scoring rules, and job process scoring rules. The trajectory recognition unit is used to acquire a trajectory recognition model based on multimodal data, analyze the preprocessed information based on the trajectory recognition model, and generate trajectory recognition results. The scoring unit is used to score based on the preset scoring rules and the trajectory recognition results, and generate a scoring result; The preprocessing of the multimodal monitoring information to obtain preprocessed information includes: Perform trajectory optimization processing on the movement trajectory information to obtain the processed trajectory; The image monitoring information is subjected to enhancement and correction processing to obtain a processed image; Perform noise reduction processing on the sound monitoring information to obtain the processed sound; The text information is sequentially subjected to text denoising, text cleaning, word segmentation and part-of-speech tagging to obtain the processed text; Preprocessed information is generated based on the processed trajectory, the processed image, the processed sound, and the processed text; The step of analyzing the preprocessed information based on the trajectory recognition model to generate trajectory recognition results includes: Multimodal feature extraction is performed on the preprocessed information to obtain multimodal data features, which are related to the movement trajectory of the robotic arm; The multimodal data features are fused to obtain fused features; Based on the trajectory recognition model, multimodal analysis is performed on the fused features to generate trajectory recognition results; The step of scoring based on the preset scoring rules and the trajectory recognition results to generate a scoring result includes: Obtain the operation program corresponding to the robotic arm operation; The job program is analyzed based on the program scoring rules to generate a first score; The trajectory recognition results are analyzed based on the work process scoring rules to generate a second score; The system acquires monitoring information on the completion of the robotic arm operation, analyzes the monitoring information based on the operation result scoring rules, and generates a third score. A scoring result is generated based on the first score, the second score, and the third score; The trajectory recognition unit is also used for: If the trajectory recognition result indicates the existence of a trajectory anomaly, the anomaly category of the trajectory recognition result is determined based on the processed information; Early warning information is generated based on the anomaly category; Based on the trajectory recognition results and the anomaly category, a preset reference answer is obtained; Output the warning information and the preset reference answer.

6. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the method described in any one of claims 1-4.

Images