Automatic scoring method and device for mechanical arm operation work of students and storage medium
By automatically identifying multimodal monitoring information by machine and combining the trajectory recognition model of GAN network, the blind spots and injustice problems of manual evaluation in traditional robotic arm operation courses are solved, and a more fair and efficient automated scoring is achieved, which improves scoring accuracy.
Patent Information
- Application Number
- CN202411268441.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-11
AI Technical Summary
In traditional robotic arm operation courses, teachers have manual monitoring and evaluation risks of regulatory blind spots and injustice, making it difficult to achieve fair and efficient automated scoring.
Multimodal monitoring information (mobile trajectory, image, sound, text) is obtained through automatic machine recognition, and analysis is performed through preprocessing and trajectory recognition model based on GAN network, and scoring results are generated in combination with preset scoring rules to achieve automated scoring.
It improves the scoring fairness and efficiency of robotic arm operation, reduces subjective influence, and improves the scoring accuracy.
Smart Images

Figure CN119090355A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an automatic scoring method for students' robot arm operation operations, an automatic scoring device for students' robot arm operation operations, and a computer-readable storage medium. Background Art
[0002] With the rapid development of information technology, the education field is increasingly inclined to adopt online education models to provide a more flexible and convenient way of learning. However, traditional practical courses, especially those involving complex skills such as robotic arm operation, still face challenges in supervision and evaluation.
[0003] In traditional hands-on courses, students are able to learn and master specific skills and operating procedures by operating the robotic arm in person. This practical training not only helps students understand the challenges and requirements in the actual working environment, but also develops their hand-eye coordination and problem-solving skills.
[0004] However, robot arm operation involves multiple skill levels, including programming, operation, and debugging. Traditional assessment methods often rely on manual monitoring by teachers, which poses risks of blind spots in supervision and unfair assessment. Therefore, a method that can automatically monitor and evaluate student operations is needed to improve the fairness and efficiency of assessment. Summary of the invention
[0005] In order to overcome the above-mentioned technical problems existing in the prior art, the embodiments of the present invention provide a method, device and storage medium for automatic scoring of students' robotic arm operation operations. By improving the traditional scoring method, on the one hand, machine automatic recognition is used for analysis and scoring, which improves fairness and scoring efficiency; on the other hand, scoring is performed through multi-dimensional monitoring data, which improves scoring accuracy.
[0006] In order to achieve the above-mentioned purpose, an embodiment of the present invention provides an automatic scoring method for students' robotic arm operation operations, the method comprising: obtaining multimodal monitoring information for the robotic arm operation operations; preprocessing the multimodal monitoring information to obtain preprocessed information; obtaining preset scoring rules; obtaining a trajectory recognition model based on multimodal data, analyzing the preprocessed information based on the trajectory recognition model, and generating a trajectory recognition result; scoring based on the preset scoring rules and the trajectory recognition result to generate a scoring result.
[0007] Preferably, the multimodal monitoring information includes movement trajectory information, image monitoring information, sound monitoring information and text information, and the preprocessing of the multimodal monitoring information to obtain preprocessed information includes: performing trajectory optimization processing on the movement trajectory information to obtain a processed trajectory; performing enhancement processing and correction processing on the image monitoring information to obtain a processed image; performing sound denoising processing on the sound monitoring information to obtain a processed sound; performing text denoising, text cleaning, word segmentation and part-of-speech tagging processing on the text information in sequence to obtain a processed text; and generating preprocessed information based on the processed trajectory, the processed image, the processed sound and the processed text.
[0008] Preferably, performing trajectory optimization processing on the movement trajectory information to obtain a processed trajectory includes: extracting multiple trajectory data points of the robotic arm based on the movement trajectory information; performing a linear fitting operation on all trajectory data points to generate a fitted trajectory; obtaining a projection distance between each trajectory data point and the fitted trajectory; determining an abnormal distance whose projection distance is greater than a preset distance threshold; and performing smoothing processing on the corresponding trajectory data points based on the abnormal distance to obtain a processed trajectory.
[0009] Preferably, the method also includes: before obtaining the trajectory recognition model based on multimodal data, obtaining an initial GAN network; obtaining historical trajectory data and corresponding historical multimodal data of the robot arm operation operation; adjusting the weight of each data in the historical multimodal data based on a preset representation learning model to generate an adjusted weight; generating a training data set based on the historical trajectory data, the historical multimodal data and the adjusted weight; training the initial GAN network based on the training data set to generate a trajectory recognition model based on multimodal data.
[0010] Preferably, the weight of each data in the historical multimodal data is adjusted based on the preset representation learning model to generate the adjusted weight, including: performing importance analysis on each data in the historical multimodal data based on the preset representation learning model to generate the most important data, the second most important data and the remaining data; respectively determining the first weight of the most important data, the second weight of the second most important data and the third weight of the remaining data; generating the adjusted weight based on the first weight, the second weight and the third weight.
[0011] Preferably, the analyzing the preprocessed information based on the trajectory recognition model to generate a trajectory recognition result includes: performing multimodal feature extraction on the preprocessed information to obtain multimodal data features, wherein the multimodal data features are related to the movement trajectory of the robotic arm; performing feature fusion on the multimodal data features to obtain fused features; and performing multimodal analysis on the fused features based on the trajectory recognition model to generate a trajectory recognition result.
[0012] Preferably, the preset scoring rules include program scoring rules, job result scoring rules and job process scoring rules. The scoring based on the preset scoring rules and the trajectory recognition result to generate a scoring result includes: obtaining the job program corresponding to the robot arm operation job; analyzing the job program based on the program scoring rules to generate a first score; analyzing the trajectory recognition result based on the job process scoring rules to generate a second score; obtaining the result monitoring information of the robot arm operation job, analyzing the result monitoring information based on the job result scoring rules to generate a third score; and generating a scoring result based on the first score, the second score and the third score.
[0013] Preferably, the method further includes: when the trajectory recognition result is characterized as having a trajectory abnormality, determining the abnormality category of the trajectory recognition result based on the processed information; generating warning information based on the abnormality category; obtaining a preset reference answer based on the trajectory recognition result and the abnormality category; and outputting the warning information and the preset reference answer.
[0014] On the other hand, the present invention also provides an automatic scoring device for students' robot arm operation operations, the device comprising: an information acquisition unit, used to acquire multimodal monitoring information for the robot arm operation operations; a preprocessing unit, used to preprocess the multimodal monitoring information to obtain preprocessed information; a rule acquisition unit, used to acquire preset scoring rules; a trajectory recognition unit, used to acquire a trajectory recognition model based on multimodal data, analyze the preprocessed information based on the trajectory recognition model, and generate a trajectory recognition result; a scoring unit, used to score based on the preset scoring rules and the trajectory recognition result, and generate a scoring result.
[0015] On the other hand, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the method provided by the present invention when the program is executed by a processor.
[0016] Through the technical solution provided by the present invention, the present invention has at least the following technical effects: By adopting a computer-based recognition method to analyze and score students' robotic arm operation assignments, they can be scored in a more objective and fair manner, which improves fairness. At the same time, teachers are no longer required to manually observe each operation assignment in real time. Instead, multimodal data is used to conduct a multi-dimensional comprehensive score for the operation assignments, which improves scoring efficiency, reduces subjective influences, and improves scoring accuracy.
[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:
[0019] Figure 1 is a specific implementation flow chart of the automatic scoring method for students' robot arm operation homework provided by an embodiment of the present invention;
[0020] Figure 2 It is a specific implementation flow chart of preprocessing multimodal monitoring information to obtain preprocessed information provided by an embodiment of the present invention;
[0021] Figure 3 is a schematic diagram of a trajectory recognition model based on multimodal data provided by an embodiment of the present invention;
[0022] Figure 4 It is a schematic diagram of the structure of an automatic scoring device for students' robot arm operation tasks provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.
[0024] The terms "system" and "network" in the embodiments of the present invention can be used interchangeably. "Multiple" means two or more than two. In view of this, "multiple" can also be understood as "at least two" in the embodiments of the present invention. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / ", unless otherwise specified, generally indicates that the previous and subsequent associated objects are in an "or" relationship. In addition, it should be understood that in the description of the embodiments of the present invention, the words "first", "second", etc. are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.
[0025] See also Figure 1 , an embodiment of the present invention provides an automatic scoring method for students' robot arm operation homework, the method comprising: S10) obtaining multi-modal monitoring information for the robot arm operation; S20) preprocessing the multimodal monitoring information to obtain preprocessed information; S30) obtaining a preset scoring rule; S40) acquiring a trajectory recognition model based on multimodal data, analyzing the pre-processed information based on the trajectory recognition model, and generating a trajectory recognition result; S50) Scoring is performed based on the preset scoring rule and the trajectory recognition result to generate a scoring result.
[0026] In one possible implementation, first, for the multimodal monitoring information of the robot arm operation, in this embodiment, a camera set in the operation scene of the robot arm operation can be used to collect multimodal monitoring information during the operation process. The multimodal monitoring information includes but is not limited to images, sounds and other messages during the operation process. At the same time, other data is also obtained through the API interface. For example, by connecting to the school's education system API interface, the student's student number, name, real-time test data, and the robot's alarm log and program code and other information are obtained, all of which are used as the above-mentioned multimodal monitoring information.
[0027] After obtaining the above multimodal monitoring information, the original information may have certain deviations due to interference and other reasons. Therefore, in order to improve the accuracy of subsequent analysis and scoring, it is necessary to perform preprocessing operations on it to obtain more accurate multimodal monitoring information.
[0028] See also Figure 2 In an embodiment of the present invention, the multimodal monitoring information includes movement trajectory information, image monitoring information, sound monitoring information and text information, and the preprocessing of the multimodal monitoring information to obtain preprocessed information includes: S21) performing trajectory optimization processing on the movement trajectory information to obtain a processed trajectory; S22) performing enhancement processing and correction processing on the image monitoring information to obtain a processed image; S23) performing sound denoising processing on the sound monitoring information to obtain processed sound; S24) performing text denoising, text cleaning, word segmentation and part-of-speech tagging processing on the text information in sequence to obtain a processed text; S25) generating pre-processed information based on the processed trajectory, the processed image, the processed sound and the processed text.
[0029] In one possible implementation, the movement trajectory information in the multimodal information is first optimized. For example, in this embodiment, before controlling the robotic arm to perform an operation, the student inputs a control program written by himself on a designated platform of the education system. The platform flashes the control program to the designated robotic arm main control unit, and then controls the robotic arm to perform the corresponding operation.
[0030] In a specific embodiment, the operation is to move a pre-set pallet to a specified position by controlling a robotic arm. During the movement, the robotic arm moves the corresponding trajectory according to a control program. During the collection of multimodal monitoring information, its movement trajectory information is used as one of the multimodal monitoring information. For example, the movement point data of the robotic arm can be read through the above-mentioned API to constitute its movement trajectory information, or the movement point data of the robotic arm can be generated by a camera according to a real-time monitoring screen to constitute the above-mentioned movement trajectory information.
[0031] In an embodiment of the present invention, performing trajectory optimization processing on the mobile trajectory information to obtain a processed trajectory includes: extracting multiple trajectory data points of the robotic arm based on the mobile trajectory information; performing a linear fitting operation on all trajectory data points to generate a fitting trajectory; obtaining a projection distance between each trajectory data point and the fitting trajectory; determining an abnormal distance whose projection distance is greater than a preset distance threshold; and smoothing the corresponding trajectory data points based on the abnormal distance to obtain a processed trajectory.
[0032] Specifically, multiple trajectory data points of the robot arm are extracted according to the above-mentioned moving trajectory information. In order to determine which abnormal points and noise points exist in the above-mentioned trajectory data points, a linear fitting operation is performed on all trajectory data points, and a fitting trajectory is generated. At this time, the projection distance between each trajectory data point and the fitting trajectory is further obtained, that is, the distance between each trajectory data point and the actual operation trajectory of the robot arm is determined. At this time, an abnormal distance greater than a preset distance threshold is further obtained from the above-mentioned projection distance. For example, in this embodiment, the step length between each two adjacent trajectory data points is 1mm, and the preset distance threshold is set to 3mm. When the distance between a certain point and the fitting trajectory exceeds 3mm, it can be determined that the point is an abnormal trajectory point. At this time, the trajectory point is smoothed, such as restoring it to the projection point of the point on the fitting trajectory, thereby realizing the optimization processing of the original trajectory data point and obtaining the processed trajectory.
[0033] In actual application, when an optical camera is used to capture the moving trajectory of the robot arm, some abnormal points will be generated, resulting in the formed moving trajectory not being consistent with the actual trajectory. Under normal circumstances, the moving trajectory of the robot arm should be consistent with the trajectory set by its program, so it can be optimized and verified. In another embodiment, after obtaining the above-mentioned multiple trajectory data points, the student's control program is further obtained, and the expected moving trajectory of the robot arm is determined according to the control program. At this time, it is determined whether the distance between the above-mentioned multiple trajectory data points and the expected moving trajectory exceeds a preset value. If so, the point is determined to be an abnormal trajectory point, and it is smoothed to obtain a processed trajectory.
[0034] At this time, the image monitoring information is further enhanced and corrected, such as including but not limited to scaling, cropping, color correction and other processing to obtain a processed image. Through the above processing, the image clarity and image contrast are improved, and the tilt angle is adjusted to obtain an image that is more conducive to subsequent processing; the sound monitoring information is subjected to sound denoising. Specifically, the existing sound denoising algorithm can be used to remove the background noise in the sound monitoring information, while enhancing the sound information on the scene during the operation of the robot arm, and obtaining the processed sound, which includes but is not limited to footsteps, conversations, collisions and other sounds; and further, the text information is subjected to text denoising, text cleaning, word segmentation and part-of-speech tagging processing in sequence. For example, in this embodiment, the text information includes but is not limited to the student's program source code, operation log information and other text information. By subjecting it to the above processing, it is processed into a data format suitable for computer recognition and analysis. Finally, the processed information is generated into pre-processed information.
[0035] In an embodiment of the present invention, multimodal data is further introduced on the basis of simple image recognition, such as conversation sounds during the robot arm operation process, log information during program execution, limit warning lights triggered during the robot arm movement process, and other information, so that the execution of students' robot arm operation operations can be evaluated from multiple dimensions, providing a sufficient and comprehensive data basis for subsequent accurate scoring.
[0036] At this time, a trajectory recognition model based on multimodal data is obtained, and the preprocessed information is analyzed to generate the corresponding trajectory recognition result. Prior to this, a trajectory recognition model based on multimodal data needs to be created first.
[0037] In an embodiment of the present invention, the method also includes: before obtaining the trajectory recognition model based on multimodal data, obtaining an initial GAN network; obtaining historical trajectory data and corresponding historical multimodal data of the robot arm operation operation; adjusting the weight of each data in the historical multimodal data based on a preset representation learning model to generate an adjusted weight; generating a training data set based on the historical trajectory data, the historical multimodal data and the adjusted weight; training the initial GAN network based on the training data set to generate a trajectory recognition model based on multimodal data.
[0038] In one possible implementation, the initial GAN network is first obtained, and then the historical trajectory data and the corresponding historical multimodal data of the robot arm operation operation are obtained. In actual application, since the multimodal data contains multi-dimensional data, but not every dimension of data has the same importance, it is necessary to assign weights to them, so as to pay more attention to the most important data, and other data are used to assist it, so as to comprehensively improve the recognition accuracy of the robot arm operation operation.
[0039] In an embodiment of the present invention, the weight of each type of data in the historical multimodal data is adjusted based on a preset representation learning model to generate an adjusted weight, including: performing an importance analysis on each type of data in the historical multimodal data based on the preset representation learning model to generate the most important data, the second most important data and the remaining data; respectively determining a first weight for the most important data, a second weight for the second most important data and a third weight for the remaining data; and generating an adjusted weight based on the first weight, the second weight and the third weight.
[0040] In one possible implementation, first, an importance analysis is performed on each type of data in the historical multimodal data according to a preset representation learning model. The preset representation learning model is, for example, an existing representation learning model or a joint learning model. Through the representation learning model or the joint learning model, each type of data in the multimodal data can be integrated and weighted, and the features that are most critical to the recognition results can be determined according to the importance of different features. The data of each modality can be integrated more efficiently, so that the model can comprehensively consider information of multiple dimensions when making judgments or predictions, thereby reducing misjudgments or uncertainties caused by insufficient or missing information from a single modality, improving overall processing efficiency, and avoiding misjudgments caused by insufficient single data source.
[0041] Specifically, the multimodal data is divided into the most important data, the second most important data and the remaining data. Of course, those skilled in the art can further divide the importance level of the data according to actual needs, which should all fall within the scope of protection of the embodiments of the present invention and will not be described in detail here. After analyzing the historical multimodal data through the common representation learning model, the first weight of the most important data, the second weight of the second most important data and the third weight of the remaining data are determined respectively.
[0042] After re-determining the weight of each modal data, a training data set is generated based on the above historical trajectory data, historical multimodal data and adjusted weights, and input into the initial GAN network for training to generate a trajectory recognition model based on multimodal data, for example, see Figure 3 , which is a schematic diagram of a trajectory recognition model based on multimodal data provided by an embodiment of the present invention. Then, the pre-processed information is analyzed using the above model to generate a trajectory recognition result.
[0043] In an embodiment of the present invention, the analyzing the preprocessed information based on the trajectory recognition model to generate a trajectory recognition result includes: performing multimodal feature extraction on the preprocessed information to obtain multimodal data features, wherein the multimodal data features are related to the movement trajectory of the robotic arm; performing feature fusion on the multimodal data features to obtain fused features; and performing multimodal analysis on the fused features based on the trajectory recognition model to generate a trajectory recognition result.
[0044] In a possible implementation, students program the robot to move the blocks, and drive the robot to move the colored blocks from the original position to the target position. In this process, the movement of the robot end kit must be within the set action space, such as a semicircle with a radius of 320. During the monitoring process, multiple trajectory routes of the robot operation are recorded and analyzed by the above trajectory recognition model. For example, in this embodiment, through comparative analysis, it is determined that the movement trajectory characteristics of the robot are consistent with the expected trajectory characteristics, and the related multimodal data characteristics of the robot are also normal, so a trajectory recognition result with a normal movement trajectory is generated.
[0045] In the embodiment of the present invention, the features of the program / text include but are not limited to: the number of code lines, the number of comment lines, identifiers (such as variable names, function names, etc.), control flow structures (such as loops, conditional statements), function calls, etc.; the features of the image / video include but are not limited to color features, shape features, spatial relationship features, texture features, etc.
[0046] In the second embodiment, although the moving trajectory characteristics are determined to be consistent with the expected trajectory characteristics during the identification process of the moving trajectory, at the start of the action, it can be seen from the monitoring image of the robot arm that the starting position of the robot arm exceeds the limit, so its indicator light is red, thereby determining that its starting position exceeds the limit. At the same time, through the API interface, the starting position exceeding limit alarm information in the robot arm control software is received, so the trajectory identification result of the starting position exceeding the limit is generated.
[0047] In the third embodiment, when obtaining the moving coordinates of the robotic arm, the robotic arm is moved out of the set action space. At this time, on the one hand, the control software will issue an over-limit alarm; on the other hand, the indicator light on the robotic arm will change from green to red; on the third hand, the trajectory point of the robotic arm will be outside the set action space. Therefore, in the scoring process, the above-mentioned multimodal data is analyzed by the trajectory recognition model to generate a trajectory recognition result of the robotic arm position out of limit.
[0048] In the fourth embodiment, during the process of scoring the robot arm operation, it is monitored that the robot arm makes a stuck sound / palletizing collision sound. Therefore, in the subsequent scoring process, in the multimodal data recognition process of the robot arm operation operation, the multimodal data features are analyzed through the common representation learning model, and the sound features are determined as the most important data features, and analysis and judgment are performed to generate a preliminary analysis result of "robot arm stuck / palletizing collision". In this embodiment, although in the process of comparing the motion trajectory of the robot arm, it is found that its motion trajectory is consistent with the expected trajectory (the robot arm trajectory may be identified as a normal trajectory when the Z-axis exceeds the limit), the trajectory recognition result based on the motion trajectory is inconsistent with the above-mentioned preliminary analysis result, but according to the weight of the sound feature, the preliminary analysis result is still taken as the final trajectory recognition result to ensure the accuracy of the final scoring of the robot arm operation operation.
[0049] In an embodiment of the present invention, multi-modal data is used to perform a multi-dimensional comprehensive analysis of the trajectory recognition of the robot arm operation operation, so as to effectively solve the analysis deviation caused by a single data source. At the same time, the weight of each data in the multimodal data is dynamically adjusted in combination with the representation learning model, so as to give priority to the use of the most important data source to objectively evaluate the operation operation, thereby greatly improving the accuracy of the final analysis.
[0050] After generating the trajectory recognition result of the robot arm operation operation, the operation is further scored according to the preset scoring rules. In an embodiment of the present invention, the preset scoring rules include program scoring rules, operation result scoring rules and operation process scoring rules, and the scoring based on the preset scoring rules and the trajectory recognition result to generate the scoring result includes: obtaining the operation program corresponding to the robot arm operation operation; analyzing the operation program based on the program scoring rules to generate a first score; analyzing the trajectory recognition result based on the operation process scoring rules to generate a second score; obtaining the result monitoring information of the completion of the robot arm operation operation, analyzing the result monitoring information based on the operation result scoring rules to generate a third score; generating the scoring result based on the first score, the second score and the third score.
[0051] In a possible implementation, the teacher may pre-set scoring rules, which may include program scoring rules, homework result scoring rules, and homework process scoring rules. For example, see the following table for a specific example of scoring rules:
[0052] In the specific implementation process, firstly, the operation program corresponding to the robot arm operation operation is obtained, which is the robot arm control program uploaded by the students. Then, the operation program is analyzed according to the above-mentioned program scoring rules to generate a first score; and according to the operation process scoring rules, for example, in this embodiment, the trajectory recognition result is analyzed according to whether it is an over-limit action to generate a second score; and according to the operation result scoring rules, the result monitoring information is analyzed to generate a third score.
[0053] For example, in a specific embodiment, by preliminarily analyzing the robot arm operation, it is determined that the operation is normal, and then all features are further compared with a data set that has been pre-marked as "correct" or "out of limit", it is found that there is an action trajectory in the student's homework that is marked as "out of limit" in the data set, that is, an error is found in the homework, so the corresponding score correction is automatically performed to obtain the final score result.
[0054] In an embodiment of the present invention, by quantitatively analyzing and scoring the students' robotic arm operation operations from multiple dimensions, on the one hand, the accuracy of automatic grading of students' control operations is effectively improved, and the user experience is improved; on the other hand, the teacher's workload is greatly reduced, the teacher's work efficiency is improved, the teacher's work pressure is reduced, and actual needs are met.
[0055] In actual application, after discovering that students have problems with their operation, they also hope to receive automatic and targeted prompts or answers to their problems. However, the existing technology has not yet provided a solution to the above technical problems.
[0056] In an embodiment of the present invention, the method further includes: when the trajectory recognition result is characterized as having a trajectory abnormality, determining the abnormality category of the trajectory recognition result based on the processed information; generating warning information based on the abnormality category; obtaining a preset reference answer based on the trajectory recognition result and the abnormality category; and outputting the warning information and the preset reference answer.
[0057] In one possible implementation, the teacher can pre-classify, organize and archive the common mistakes and common problems of students. In the subsequent homework grading process, when problems are detected in the students' operation homework, the track recognition results are automatically analyzed from multiple dimensions based on multimodal data to determine the abnormal category and generate corresponding warning information. On the other hand, the corresponding preset reference answers are searched according to the above track recognition results and abnormal categories, and output to the students, so that the students can improve and correct them in time to avoid the recurrence of similar problems, thereby effectively improving the students' learning efficiency and learning effect.
[0058] See also Figure 4Based on the same inventive concept, an embodiment of the present invention further provides an automated grading device for students' robot arm operation operations, the device comprising: an information acquisition unit, used to acquire multimodal monitoring information for the robot arm operation operations; a preprocessing unit, used to preprocess the multimodal monitoring information to obtain preprocessed information; a rule acquisition unit, used to acquire preset scoring rules; a trajectory recognition unit, used to acquire a trajectory recognition model based on multimodal data, analyze the preprocessed information based on the trajectory recognition model, and generate a trajectory recognition result; a scoring unit, used to score based on the preset scoring rules and the trajectory recognition result, and generate a scoring result.
[0059] Furthermore, an embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in the present invention is implemented.
[0060] The optional implementation modes of the embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above implementation modes. Within the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical scheme of the embodiments of the present invention, and these simple modifications all belong to the protection scope of the embodiments of the present invention.
[0061] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe various possible combinations.
[0062] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a single-chip microcomputer, a chip or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0063] In addition, various implementations of the embodiments of the present invention may be arbitrarily combined, and as long as they do not violate the concept of the embodiments of the present invention, they should also be regarded as the contents disclosed in the embodiments of the present invention.
Claims
1. An automatic scoring method for students' robot arm operation homework, It is characterized in that The method comprises: Obtain multi-modal monitoring information for robot arm operation operations; Preprocessing the multimodal monitoring information to obtain preprocessed information; Get the preset scoring rules; Acquire a trajectory recognition model based on multimodal data, analyze the preprocessed information based on the trajectory recognition model, and generate a trajectory recognition result; Scoring is performed based on the preset scoring rules and the trajectory recognition result to generate a scoring result.
2. The method according to claim 1, It is characterized in that The multimodal monitoring information includes movement trajectory information, image monitoring information, sound monitoring information and text information, and the preprocessing of the multimodal monitoring information to obtain preprocessed information includes: Performing trajectory optimization processing on the movement trajectory information to obtain a processed trajectory; Performing enhancement processing and correction processing on the image monitoring information to obtain a processed image; Performing sound denoising processing on the sound monitoring information to obtain processed sound; Performing text denoising, text cleaning, word segmentation and part-of-speech tagging processing on the text information in sequence to obtain processed text; Pre-processed information is generated based on the processed trajectory, the processed image, the processed sound and the processed text.
3. The method according to claim 2, It is characterized in that The performing trajectory optimization processing on the movement trajectory information to obtain a processed trajectory includes: Extracting a plurality of trajectory data points of the robot arm based on the movement trajectory information; Perform linear fitting operations on all trajectory data points to generate fitting trajectories; Obtaining the projection distance between each trajectory data point and the fitting trajectory; Determine an abnormal distance where the projected distance is greater than a preset distance threshold; The corresponding trajectory data points are smoothed based on the abnormal distance to obtain a processed trajectory.
4. The method according to claim 1, It is characterized in that The method further comprises: Before obtaining the trajectory recognition model based on multimodal data, obtaining an initial GAN network; Acquire historical trajectory data and corresponding historical multimodal data of the robot arm operation; Adjusting the weight of each type of data in the historical multimodal data based on a preset representation learning model to generate an adjusted weight; generating a training data set based on the historical trajectory data, the historical multimodal data, and the adjusted weights; The initial GAN network is trained based on the training data set to generate a trajectory recognition model based on multimodal data.
5. The method according to claim 4, It is characterized in that The step of adjusting the weight of each type of data in the historical multimodal data based on the preset representation learning model to generate an adjusted weight includes: Performing importance analysis on each type of data in the historical multimodal data based on the preset representation learning model to generate the most important data, the second most important data and the remaining data; respectively determining a first weight of the most important data, a second weight of the less important data, and a third weight of the remaining data; An adjusted weight is generated based on the first weight, the second weight, and the third weight.
6. The method according to claim 2, It is characterized in that The analyzing the pre-processed information based on the trajectory recognition model to generate a trajectory recognition result includes: Performing multimodal feature extraction on the preprocessed information to obtain multimodal data features, wherein the multimodal data features are related to the movement trajectory of the robotic arm; Performing feature fusion on the multimodal data features to obtain fused features; Based on the trajectory recognition model, a multimodal analysis is performed on the fused features to generate a trajectory recognition result.
7. The method according to claim 1, It is characterized in that The preset scoring rules include program scoring rules, operation result scoring rules and operation process scoring rules. The scoring is performed based on the preset scoring rules and the trajectory recognition result to generate a scoring result, including: Obtaining an operation program corresponding to the operation of the robot arm; Analyze the operation procedure based on the procedure scoring rule to generate a first score; Analyzing the trajectory recognition result based on the operation process scoring rule to generate a second score; Acquire monitoring information of the result of the robot arm operation operation, analyze the monitoring information based on the operation result scoring rule, and generate a third score; A rating result is generated based on the first rating, the second rating, and the third rating.
8. The method according to claim 1, It is characterized in that The method further comprises: In a case where the trajectory recognition result is characterized as having a trajectory abnormality, determining an abnormality category of the trajectory recognition result based on the processed information; generating warning information based on the abnormality category; Obtaining a preset reference answer based on the trajectory recognition result and the abnormal category; Output the warning information and the preset reference answer.
9. An automatic scoring device for students' robot arm operation homework, It is characterized in that The device comprises: An information acquisition unit, used to acquire multi-modal monitoring information for the robot arm operation; A preprocessing unit, used to preprocess the multimodal monitoring information to obtain preprocessed information; A rule acquisition unit, used to acquire preset scoring rules; A trajectory recognition unit, used for acquiring a trajectory recognition model based on multimodal data, analyzing the preprocessed information based on the trajectory recognition model, and generating a trajectory recognition result; A scoring unit is used to score based on the preset scoring rules and the trajectory recognition result to generate a scoring result.
10. A computer storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the method described in any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
AI automatic scoring method and system
CN113435379A
Scoring method, device and equipment for chemical experiment operation and readable storage medium
CN115019240A
Operation behavior identification method and device
CN115439928A
Cross-modal sensitive information identification method
CN117668292A
Physical experiment scoring method and device and storage medium
CN118195362A
