Robot-assisted teaching method and system in teaching scene

Abstract

The invention discloses a robot-assisted teaching method and system in a teaching scene, and relates to the technical field of robot-assisted teaching. The method comprises the following steps: constructing a teaching scene map through full-scene scanning, receiving a student help signal sent by a special desk, and collecting a student video at the same time; the system determines the help-seeking behavior and sequence of the student according to the help-seeking signal and the video, assists the teaching robot to go to the student desk in sequence, and records a practice video. And the robot compares the student video with a standard video, displays errors, explains reasons and corrects a scheme by voice, supervises the students to execute and prompts until the students complete correct operation. And after completion, continuing to teach the next student until all help-seeking behaviors are taught. According to the invention, highly automatic classroom auxiliary teaching is realized, manual teaching assistance intervention is not needed, human resource investment and management cost in the teaching process are greatly reduced, timeliness and consistency of teaching are ensured, and efficient regional student classroom auxiliary teaching is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot-assisted teaching, in particular to a robot-assisted teaching method and system in a teaching scene. BACKGROUND

[0002] With the continuous promotion of elitist education, classroom teaching has higher and higher requirements for teachers. In addition to having a very sufficient theoretical knowledge, teachers also need to have the ability to combine theory with practice.

[0003] In the prior art, some courses in the part of the Sino-foreign cooperative education project or institution require teaching in a 1:1 ratio of theory and practice. Theoretical teaching can be basically handled by one teacher, but for a class with multiple students, one teacher is often very busy, and usually a student assistant is assigned to assist the teacher in teaching students to complete the practice content.

[0004] However, student assistants face some objective difficulties, such as high requirements for student assistants, who need to understand both theoretical knowledge and practical teaching, and even need to demonstrate practice to students. In addition, for courses in the Sino-foreign cooperative education project, student assistants are also required to meet the language proficiency standards. In this case, student assistants often cannot meet the course teaching requirements.

[0005] Therefore, there is an urgent need for a classroom-assisted teaching method suitable for a teaching scene to effectively assist students in completing the practice content in the classroom. SUMMARY

[0006] Therefore, it is necessary to provide a robot-assisted teaching method and system in a teaching scene to solve the above technical problems.

[0007] The present application adopts the following technical solutions: The present application provides a robot-assisted teaching method in a teaching scene, comprising: obtaining full-scene scanning information of the teaching scene to construct a teaching scene map; receiving a help signal sent by a student in the scene map through a signal button on a specially designed desk, and collecting a student video in the teaching scene map; determining a student help behavior and a generation order of the student help behavior according to the student help signal and the student video; and recording student practice videos in sequence according to the generation order of the student help behavior; comparing the student practice videos with standard practice videos to determine and display errors in the student practice operation; and prompting the causes of the errors and optional solutions to correct the errors through voice; supervising the process of the student executing the optional solutions and prompting through voice at the corrected place until the student at the current desk position completes the correct practice operation; After the current student practice operation is completed, the next student practice video in sequence is compared with the standard practice video and corrected until all students correctly complete the practice operation.

[0008] Preferably, the student help behavior is determined according to the student video, and specifically includes: Key frames at multiple time points in the student video are extracted, and a time sequence model is used to calculate the difference between the current time key frame and all frames before the current time to determine the motion change of the student hand region; According to the motion change of the student hand region, a target detection model is used to detect the student hand region and the help signal button region of the characteristic desk to determine whether the student has a help behavior, including: If the student hand region covers the help signal button region of the characteristic desk, and the robot receives the help signal, it is determined that the student has a help behavior; If the student hand motion is that the fingertips are upward and maintained for more than 3 seconds, it is determined that the student is raising his hand, and the student has a help behavior; If it is other conditions, it is determined that the student does not have a help behavior.

[0009] Preferably, before sequentially recording the student practice video according to the sequence of the student help behavior, it further includes: Determine whether the student's desk position to be visited has a teacher teaching the student; If no teacher is teaching, go to the student's desk position; If a teacher is teaching, terminate the visit to the student's desk position, and reorder the sequence of the student help behavior.

[0010] Preferably, the sequence of the student help behavior is reordered, specifically including: The student number and desk number corresponding to the student desk position of the teacher are excluded to obtain a new sequence of student help behavior; If a new help signal is received during the judgment process, the student number and desk number are added to the end of the sequence of student help behavior to obtain the reordered sequence of student help behavior.

[0011] Preferably, if the teacher completes the teaching of the student seeking help, the student is prompted to send a termination help signal by double-clicking the button on the desk to automatically cancel the help behavior of the student.

[0012] Preferably, the method further includes: According to the course time, a course end instruction is sent in advance, and after all students complete the practice operation, the auxiliary teaching robot goes to the charging area for charging.

[0013] The application discloses a robot-assisted teaching system in a teaching scene, comprising: an auxiliary teaching robot, a central control platform, a small server and a special desk. The auxiliary teaching robot is used for acquiring full-scene scanning information of the teaching scene to construct a teaching scene map, recording student practice videos in sequence according to the generation order of student help-seeking behaviors, comparing the student practice videos with standard practice videos to determine and display errors in student practice operations, and prompting the causes of the errors and optional solutions for correcting the errors through voice, supervising the process of the students executing the optional solutions and giving voice prompts at the corrected positions until the students at the current desk position complete correct practice operations. The central control platform is used for receiving help-seeking signals sent by students in the scene map through signal buttons on the special desk and collecting student videos in the teaching scene map. The small server is used for determining student help-seeking behaviors and the generation order of the student help-seeking behaviors according to the student help-seeking signals and the student videos. The special desk is used for sending the student help-seeking signals.

[0014] The application further provides a computer readable storage medium, the storage medium storing a computer program, and the computer program is executed by a processor to realize the robot-assisted teaching method in the teaching scene.

[0015] The application further provides a computer device, characterized in comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor realizes the robot-assisted teaching method in the teaching scene when executing the program.

[0016] The application provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor realizes the robot-assisted teaching method in the teaching scene when executing the program.

[0017] The application adopts the above at least one technical scheme to achieve the following beneficial effects: In the robot-assisted teaching method in a teaching scene provided by the application, the complete teaching scene map can be accurately constructed and stored through efficient full-scene scanning of the teaching scene, the workload of repeated scanning is greatly reduced, the same scene map can be shared between different robots, and the utilization efficiency of resources and the convenience of deployment are significantly improved; based on the help-seeking signals and behavior video data of students, the help-seeking behaviors of students can be intelligently identified, and the generation order of multiple help-seeking events can be accurately determined, so that intelligent sorting of task processing is realized; according to the identified help-seeking order, the path can be autonomously planned and the target student desk position can be sequentially reached, the practical operation video of the student can be automatically recorded, and intelligent comparison and analysis can be performed with the standard or historical operation, so that the specific error points in the practical operation of the student can be accurately identified and intuitively displayed; the reason for the error and the optional correction scheme for teaching the student to complete the practice are explained by voice, without manual operation, and the cost of human assistant teaching is reduced.

[0018] In summary, through the robot-assisted teaching method and system provided by the application, the entire process from scene construction, help-seeking identification, response sorting, path planning, on-site recording, operation comparison, error diagnosis to voice teaching is highly automated, completely without manual assistant intervention, greatly reducing the human resource investment and management cost in the teaching process, while ensuring the timeliness and consistency of teaching, and realizing efficient regional student classroom auxiliary teaching. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the principles of the application, and do not limit the application in any way. In the drawings:

[0020] Figure 1 A flowchart of a robot-assisted teaching method in a teaching scene provided by the application; Figure 2 An auxiliary teaching robot schematic diagram of a robot-assisted teaching method in a teaching scene provided by the application; Figure 3 An automatic identification of student help-seeking behavior flowchart of a robot-assisted teaching method in a teaching scene provided by the application; Figure 4 A robot-assisted teaching system schematic diagram provided by the application in a teaching scene; Figure 5 A computer device schematic diagram of a robot-assisted teaching method in a teaching scene provided by the application. DETAILED DESCRIPTION

[0021] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments of the present invention and corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in the specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0023] Figure 1 The following is a flow chart of a robot-assisted teaching method in a teaching scenario according to the present invention, which specifically includes the following steps: S101: Obtain full-scene scanning information of the teaching scene to construct a teaching scene map.

[0024] S102: Receive help signals sent by students in the scene map through the signal buttons on the special desks, and collect student videos in the teaching scene map; determine the student help-seeking behavior and the order in which the student help-seeking behavior occurs based on the student help-seeking signals and student videos; record the student practice videos in sequence according to the order in which the student help-seeking behavior occurs.

[0025] Optionally, determining a student's request for help based on a student video specifically includes: extracting key frames at multiple moments in the student video, calculating the difference between the current key frame and all previous frames using a time series model, and determining changes in the student's hand area; based on these changes in hand area, using an object detection model, detecting the student's hand area and the help button area of ​​the special desk to determine whether the student has requested help. This includes: if the student's hand area covers the help button area of ​​the special desk and the robot receives a help signal, determining that the student has requested help; if the student's hand movement is with the fingertips pointing upwards and maintained for more than three seconds, determining that the student is raising their hand and requesting help; otherwise, determining that the student has not requested help.

[0026] Specifically, the physical display image is transformed into HSV color space, contour features are extracted and Hu moment matching is performed with the standard template; optical flow field analysis is performed on the practical operation video stream, and the dynamic time warping distance between the action trajectory and the standard action sequence is calculated; when the Hu moment deviation value or DTW distance exceeds the preset tolerance range, an analysis report containing the error position coordinates and deviation value is generated.

[0027] Optionally, before going to the student's desk position in sequence and recording the student practice video, it further comprises: judging whether the student's desk position to be arrived at has a teacher teaching the student; if not, going to the student's desk position; if yes, terminating going to the student's desk position, and reordering the sequence of the student's seeking help behavior.

[0028] Optionally, the sequence of the student's seeking help behavior is reordered, specifically comprising: excluding the student number and the desk number corresponding to the teacher's student desk position to obtain a new sequence of the student's seeking help behavior; if the robot receives a new help signal during the judgment process, adding the student number and the desk number to the end of the sequence of the student's seeking help behavior to obtain a reordered sequence of the student's seeking help behavior.

[0029] Specifically, assuming that there are N students seeking help, the sequence before reordering is 1,…,N, and the corresponding desk numbers are D1,…,DN. N If the student seeking help number currently accepted by the robot-assisted teaching is i (1≤i≤N), after the robot completes the assistance teaching, the desk number D i is removed; after the student with the seeking help number i completes the assistance teaching, the robot observes the student experiment in the fixed area and detects the teacher's position, if the teacher is at the student's desk, and the student's seeking help number is j, and the desk number is D j , then the desk number D j is also removed; when there are m students sending new help signals, their numbers will be sequentially added to the help sequence, denoted as N+1,…,N+n, and the corresponding desk numbers are D N+1 ,…,DN+n. N+m The new help sequence is changed to D1,…,D i-1 ,D i+1 ,…,D j-1 ,D j+1 ,…,D N ,D N+1 ,…,D N+m .

[0030] S103: comparing the student practice video with the standard practice video to determine and display the errors in the student's practice operation; and prompting the causes of the errors and the optional solutions to correct the errors through voice; supervising the student's execution of the optional solution and giving voice prompts at the correction, until the student at the current desk position completes the correct practice operation.

[0031] Optionally, during the teaching process, if the teacher has completed teaching the student who is asking for help, the auxiliary teaching robot requires the student to double-click the button on the desk to send a termination signal for asking for help to the auxiliary teaching robot, automatically canceling the student's request for help.

[0032] Specifically, the teaching assistant robot can use video analysis algorithms (such as object detection algorithms combined with long-short-term memory networks) to detect and extract key student actions during practice, performing frame-by-frame comparison analysis. From the accuracy of key actions to the sequence of steps, every detail is accurately captured. By comparing student videos with standard demonstration videos, the robot can immediately identify incorrect student actions and present them to the student with a striking and easy-to-understand graphical interface, such as a red box to mark incorrect actions and an arrow to indicate incorrect operation directions. For voice explanations, the robot uses a variety of specialized voice templates to select the most appropriate explanation method based on the specific practice content and error type, explaining the cause of the error in clear and concise language. For example, "In a chemistry experiment, you added reagents in the opposite order from the standard procedure. This may prevent the reaction from proceeding properly or even produce dangerous side reactions." "When assembling a mechanical model, you omitted a key connection, which is the main reason for the model's instability." The robot also provides students with multiple options for correcting errors, each with detailed steps and precautions. For example, "Option 1: Re-adjust the order of reagent addition according to the standard video; Option 2: Separate the mixed reagents first and then re-mix them in the correct order"; "Option 1: Disassemble the assembled parts and reinstall the connecting components; Option 2: Use spare connecting parts for adjustment and reinforcement." As students execute the corrections, the robot monitors the process, its camera tracking the student's movements in real time. If it detects any deviations, it immediately issues voice prompts, such as "Please note, your hand is at the wrong angle; it should be tilted upwards 30 degrees" or "Please apply gentle pressure in this step, otherwise it will cause damage," until the student has perfectly completed the correct procedure. Regarding the "termination" signal for help, when the instructor completes the instruction, the teaching assistant robot prompts the student, via voice or screen, to double-click a button on the desk. This button is conveniently located on the desk. With a simple press, the student receives the signal and automatically cancels the student's request for help, freeing them to focus on other students in need, ensuring a highly efficient and coherent teaching process.

[0033] S104: After the current desk student completes the practical operation, the next student's practice video is compared with the standard practice video and corrected until all the students seeking help complete the practical operation correctly.

[0034] Optionally, the teaching teacher sends a course end instruction to the auxiliary robot in advance according to the course time, and after completing the teaching of all student help behaviors, the auxiliary teaching robot goes to the charging area for charging by itself.

[0035] Specifically, once the auxiliary teaching robot completes the teaching of the current student at the desk, the built-in intelligent navigation system thereof is started quickly, and the shortest path to the next student desk in sequence is planned in combination with the electronic map pre-drawn according to the classroom layout and the student seat distribution and the help student sequence. The robot uses the bottom laser radar and the visual recognition system to perceive the surrounding environment in real time, avoids the desks and chairs, obstacles and moving student groups, and moves to the next help student desk at a uniform speed and a stable posture.

[0036] After arriving, the robot starts a new round of teaching process through a multi-modal interactive interface (such as a touch screen and a voice prompt). The binocular camera thereof automatically calibrates the shooting angle, captures the student operation picture and performs pixel-level comparison with the cloud standard video library. When it is detected that there is an improper posture in the student action (such as the angle between the soldering gun and the circuit board deviates from the standard 15°±5° range), the robot generates a three-dimensional animation model in real time and gives a voice explanation (such as “please support the tail of the soldering gun at the root of the ring finger and the little finger, so that the tool and the circuit board form a 15-degree angle, which can avoid uneven soldering”).

[0037] In the supervision correction stage, the robot calls the posture action deep learning algorithm to score the real-time correction of the student's action, and when the score reaches 90 points or more for three times in a row, the system automatically determines that the operation is qualified. If the teaching teacher sends a course end instruction (including a classroom summary PPT and a homework two-dimensional code) through a mobile terminal APP 10 minutes in advance, the auxiliary robot will preferentially complete the teaching of the last student, and after confirming that all help signals have been cleared, it will automatically switch to a low-power return mode. The magnetic base thereof will be precisely connected with the preset charging dock, so as to ensure that the charging contact error is controlled within ±0.5 mm, and the state report feedback (such as “charging is completed, power is 98%, available time for next class is 6 hours and 24 minutes”) is fed back to the teacher terminal through the wireless communication module, so as to realize the closed-loop management and intelligent operation and maintenance of the whole process of auxiliary teaching.

[0038] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the range disclosed by the present application.

[0039] The above is a robot-assisted teaching method in a teaching scene according to one or more embodiments of the present application. Based on the same idea, the present application also provides a corresponding robot-assisted teaching system in a teaching scene, such as Figure 4As shown.

[0040] Figure 4 A schematic diagram of a robot-assisted teaching system in a teaching scene is provided in the present application, and the system comprises: A robot-assisted teaching system in a teaching scene comprises an auxiliary teaching robot, a central control platform, a small server and a specially designed desk. The auxiliary teaching robot is configured to acquire full-scene scanning information of the teaching scene to construct a teaching scene map, determine a student help-seeking behavior and a generation sequence of the student help-seeking behavior according to a student help-seeking signal and a student video, sequentially record student practice videos according to the generation sequence of the student help-seeking behavior, compare the student practice videos with standard practice videos to determine and display errors in student practice operations, and prompt reasons for the errors and optional solutions to correct the errors through voice, supervise a process in which the student executes the optional solutions and provide voice prompts at the corrected positions until the student at the current desk position completes correct practice operations. The central control platform is configured to receive a student help-seeking signal sent by a signal button on the specially designed desk in the scene map and collect student videos in the teaching scene map. The small server is configured to determine a student help-seeking behavior and a generation sequence of the student help-seeking behavior according to a student help-seeking signal and a student video. The specially designed desk is configured to send a student help-seeking signal.

[0041] For specific limitations of the robot-assisted teaching system in a teaching scene, refer to the limitations of the robot-assisted teaching method in a teaching scene as described above, which will not be repeated here. The modules in the robot-assisted teaching system in a teaching scene described above can be realized by software, hardware and combinations thereof in whole or in part. The modules described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform the operations corresponding to the modules.

[0042] The present application also provides a computer readable storage medium, which stores a computer program, and the computer program can be used to execute the robot-assisted teaching method in a teaching scene as described above. Figure 1 The present application also provides a robot-assisted teaching method in a teaching scene.

[0043] The present application also provides a computer device as shown in the structure schematic diagram, which comprises: Figure 5 The present application also provides a computer device as shown in the structure schematic diagram, which comprises: Figure 5 As shown in the structure schematic diagram, the computer device comprises a processor, an internal bus, a network interface, a memory and a non-volatile memory, and of course can also comprise other hardware required by a business. The processor reads corresponding computer programs from the non-volatile memory into the memory and then runs to realize the robot-assisted teaching method in a teaching scene as described above. Figure 1Provided is a robot-assisted teaching method in a teaching scene.

[0044] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments of the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

Claims

1. A robot-assisted teaching method in a teaching scenario, applied to a robot-assisted teaching system in a teaching scenario, characterized in that: include: Obtain full scene scanning information of the teaching scene to build a teaching scene map; Receive help signals sent by students in the scene map through the signal button on the special desk, and collect student videos in the teaching scene map; determine the student's help-seeking behavior and the order in which the student's help-seeking behavior occurs based on the student's help-seeking signal and student video; record the student's practice video in sequence according to the order in which the student's help-seeking behavior occurs; Compare the student practice video with the standard practice video to identify and display errors in the student's practice operation; and provide voice prompts to indicate the cause of the error and the options for correcting the error; supervise the student's implementation of the options and provide voice prompts at the correction points until the student at the current desk position completes the correct practice operation; After the current desk student completes the practical operation, the next student's practice video will be compared with the standard practice video and corrected until all the students seeking help complete the practical operation correctly.

2. The robot-assisted teaching method in a teaching scenario according to claim 1, characterized in that: Determine student help-seeking behaviors based on student videos, including: Extract key frames at multiple moments in the student video, and use the timing model to calculate the difference between the current key frame and all frames before the current moment to determine the movement changes in the student's hand area; Based on the changes in the student's hand movements, the target detection model is used to detect the student's hand area and the help signal button area on the special desk to determine whether the student has requested help, including: If the student's hand area covers the help signal button area of ​​the special desk and the robot receives the help signal, it is determined that the student has asked for help; If the student's hand movement is with the fingertips pointing upwards and is maintained for more than 3 seconds, it is determined that the student is raising his hand and is asking for help; If it is any other case, it is determined that the student does not engage in help-seeking behavior.

3. The robot-assisted teaching method in a teaching scenario according to claim 1, characterized in that: Before recording the student practice videos in sequence according to the order in which the student's help-seeking behavior occurs, the method further includes: Determine whether there is a teacher teaching the student at the desk where the student is about to go; If there is no teacher teaching, go to the student's desk; If a teacher is currently teaching, the process of going to the student's desk will be terminated, and the order in which students' requests for help are reordered.

4. The robot-assisted teaching method in a teaching scenario according to claim 3, characterized in that: The reordering of the order in which students' help-seeking behaviors occur specifically includes: Eliminate the student numbers and desk numbers corresponding to the teacher's desk position to obtain the new order of student help-seeking behaviors; If a new help-seeking signal is received during the judgment process, the student number and desk number are added to the end of the generation sequence of the student help-seeking behavior to obtain the re-ordered generation sequence of the student help-seeking behavior.

5. The robot-assisted teaching method in a teaching scenario according to claim 3, characterized in that: If the teacher has finished teaching the student who asked for help, the teacher will prompt the student to double-click the button on the desk to send a termination signal for asking for help, automatically canceling the student's asking for help behavior.

6. The robot-assisted teaching method in a teaching scenario according to claim 1, characterized in that: The method further comprises: The course end instruction is sent in advance according to the course time. After all students have completed the practical operation, the auxiliary teaching robot will go to the charging area to charge by itself.

7. A robot-assisted teaching system in a teaching scenario, characterized in that: include: Auxiliary teaching robots, central control platforms, small servers, and special desks; The auxiliary teaching robot is used to obtain full-scene scanning information of the teaching scene to construct a teaching scene map; determine the student's help-seeking behavior and the order in which the student's help-seeking behavior occurs based on the student's help-seeking signal and the student video; record the student's practice video in sequence according to the order in which the student's help-seeking behavior occurs; compare the student's practice video with the standard practice video, determine and display errors in the student's practice operation; and provide voice prompts to the cause of the error and optional solutions for correcting the error; supervise the student's execution of the optional solution and provide voice prompts at the correction point until the student at the current desk position completes the correct practice operation; The central control platform is used to receive help signals sent by students in the scene map through the signal buttons on the special desks, and collect student videos in the teaching scene map; The small server is used to determine the student's help-seeking behavior and the order in which the student's help-seeking behavior occurs based on the student's help-seeking signal and the student's video; The specially made desk is used to send help signals to students.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images