Ros robot intelligent teaching auxiliary system, method, equipment, medium and product based on large model

Abstract

The application discloses a ROS robot intelligent teaching auxiliary system and method based on a large model, equipment, a medium and a product, relates to the teaching auxiliary field, and the system comprises a user input module, which is used for selecting an input mode according to a user use scene and inputting text information according to the input mode; a large model module, which is used for generating user required text information based on a large model according to the text information and large model prompt information; a keyword matching module, which is used for performing keyword matching according to the user required text information, and determining keyword matching information; and a ROS robot module, which is used for controlling a ROS robot to execute a corresponding ROS program according to the keyword matching information, and dynamically modifying parameter information of the ROS program, so as to assist in completing a teaching task. The application can comprehensively assist teachers in guiding students to quickly understand teaching content.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of teaching aids, in particular to a ROS robot intelligent teaching aid system and method based on a large model, equipment, medium and product. BACKGROUND

[0002] In recent years, with the rise of large models, large model technology with natural language generation as the core can directly generate knowledge questions and answers, design models, code snippets and other results through natural language description requirements, meeting the needs of various industries.

[0003] In the field of teaching, the current teaching method is essentially teacher-led and student-centered, with in-class teaching and after-school practice. The effectiveness of teaching is limited by the teaching input, professional level, course design and software resources of the teacher. Before the theoretical teaching link, the teacher spends a lot of time designing teaching cases; during the theoretical teaching, the teacher's teaching content is limited to the content designed in advance, including PPT, exercises, etc.; at the same time, the static display of teaching content cannot vividly explain the theory and rules of robot engineering algorithms. It can be seen that the current teaching method has low teacher preparation efficiency and poor interest and practicality in classroom teaching, and cannot fully assist teachers in guiding students to quickly understand the teaching content. SUMMARY

[0004] The purpose of the present application is to provide a ROS robot intelligent teaching aid system and method based on a large model, equipment, medium and product, to solve the problem that teachers cannot fully assist students in quickly understanding the teaching content.

[0005] To achieve the above-mentioned purpose, the present application provides the following solutions:

[0006] In a first aspect, the present application provides a ROS robot intelligent teaching aid system based on a large model, comprising:

[0007] A user input module for selecting an input mode according to a user usage scenario, and inputting text information according to the input mode; the input mode includes a voice recognition text input mode and a keyboard input mode;

[0008] A large model module for generating user required text information based on a large model according to the text information and large model prompt information; the large model prompt information includes theme, target, background information, role setting, input and output prompt, and restrictions and taboos;

[0009] A keyword matching module for keyword matching according to the user required text information to determine keyword matching information;

[0010] The ROS robot module is configured to control the ROS robot to execute a corresponding ROS program according to the keyword matching information, and dynamically modify parameter information of the ROS program, thereby assisting in completing the teaching task.

[0011] In a second aspect, the present application provides a ROS robot intelligent teaching assistance method based on a large model, comprising:

[0012] According to the user usage scenario, an input mode is selected, and text information is input according to the input mode; the input mode includes a voice recognition text input mode and a keyboard input mode;

[0013] Based on a large model, user required text information is generated according to the text information and large model prompt information; the large model prompt information includes a theme, a target, background information, role setting, input and output prompts, and restricted and taboo content;

[0014] According to the user required text information, keyword matching is performed to determine keyword matching information;

[0015] According to the keyword matching information, the ROS robot is controlled to execute a corresponding ROS program, and parameter information of the ROS program is dynamically modified, thereby assisting in completing the teaching task.

[0016] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the ROS robot intelligent teaching assistance method based on a large model.

[0017] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the ROS robot intelligent teaching assistance method based on a large model.

[0018] In a fifth aspect, the present application provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the ROS robot intelligent teaching assistance method based on a large model.

[0019] According to the specific embodiments of the present application, the following technical effects are disclosed:

[0020] The application generates user required text information according to input text information and large model prompt information through a user input module, a large model module, a keyword matching module and a Robot Operating System (ROS) robot module, determines keyword matching information to control the ROS robot to execute a corresponding ROS program, and dynamically modifies parameter information of the ROS program. Since the teacher knows that the current ROS program is developed according to the principle of a Proportional-Integral-Differential Controller (PID) algorithm, the system dynamically modifies the values of the parameters P, I and D of the ROS program to dynamically teach the student how these three parameters affect the control effect of the robot. The teacher does not need to spend time inputting execution commands through a keyboard or manually searching and modifying parameters in the file system of the ROS robot, which greatly saves the teacher's time. It can be seen that the system shown in the application can enrich the teaching content of the teacher, guide the student to understand the physical meaning of the parameters and the engineering effect brought by the modification of the parameters through on-site teaching, improve the interesting and practical nature of classroom teaching, and comprehensively assist the teacher to guide the student to quickly understand the teaching content. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0022] Figure 1 A schematic diagram of a ROS robot intelligent teaching auxiliary system based on a large model provided by an embodiment of the present application;

[0023] Figure 2 A schematic diagram of an application scene of a ROS robot intelligent teaching auxiliary system based on a large model provided by an embodiment of the present application;

[0024] Figure 3 A flowchart of a ROS robot intelligent teaching auxiliary method based on a large model provided by an embodiment of the present application;

[0025] Figure 4 A schematic diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] With reference to the drawings and specific embodiments described below, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0027] The above purposes, features and advantages of the present application will be more apparent and understandable. The present application will be further described in detail below with reference to the drawings and specific embodiments.

[0028] As shown in the Figure 1 Embodiments of the present application provide a ROS robot intelligent teaching auxiliary system based on a large model, which comprises:

[0029] A user input module is configured to select an input mode according to a user usage scenario and input text information according to the input mode. The input mode includes a voice recognition text input mode and a keyboard input mode.

[0030] A large model module is configured to generate user required text information based on a large model according to the text information and large model prompt information. The large model prompt information includes a theme, a target, background information, role setting, input and output prompts, and restricted and taboo content. The large model can be ChatGPT.

[0031] A keyword matching module is configured to perform keyword matching according to the user required text information to determine keyword matching information.

[0032] A ROS robot module is configured to control a ROS robot to execute a corresponding ROS program according to the keyword matching information and dynamically modify parameter information of the ROS program to assist in completing a teaching task.

[0033] In an exemplary embodiment, the user input module specifically comprises:

[0034] A keyboard input unit is configured to input text information in a keyboard input mode.

[0035] A voice recognition unit is configured to recognize voice information and convert the voice information into text information in the voice recognition text input mode.

[0036] A text information modification unit is configured to modify the text information converted by the voice recognition unit in the keyboard input mode.

[0037] Further, when the system is used for teacher teaching case design, keyboard input fits the current scene; when the system is used for teacher teaching link, the teacher can be in any position in the classroom, therefore, the teacher can obtain the teacher voice through the microphone, and then convert the teacher voice information into text information through the voice recognition unit, and since the classroom environment can be relatively noisy, this method also supports keyboard modification of voice recognition information results.

[0038] In an exemplary embodiment, the large model module specifically comprises:

[0039] The matching unit is configured to dynamically match large model prompt information according to the text information.

[0040] The user required text information generation unit is configured to input the large model prompt information into the large model to generate user required text information.

[0041] Further, the large model has randomness for the generated question and answer results, in order to make the large model output fit the current use scene, first, large model prompt information for different scenes needs to be constructed, for example, in the teaching plan design link, the large model can be told that the teaching plan prompt information is: "you are a teacher, now you need to write a teaching case, a complete teaching case includes: teaching purpose, teaching key points and difficulties, and teaching basic content, teaching basic content includes theoretical knowledge points, after-school exercises and teaching case analysis, among which theoretical knowledge points need to be explained thoroughly, core concepts need to be described; 3-5 after-school exercises of different difficulties should be given; teaching case analysis design needs to be explained thoroughly, and several different difficulty related code cases can be given, and the program principles involved are explained in detail to achieve the teaching purpose."

[0042] If in the teaching link, the large model needs to start the corresponding ROS program according to the user input, the large model prompt is: "you now have a ROS robot (i.e. role positioning), which can move omnidirectionally and can use a mechanical arm to pick up objects (i.e. background information).

[0043] The existing robot has the following programs: when performing corresponding tasks, you need to execute the corresponding program according to the ROS mechanism (i.e. topic and target).

[0044] Specific input and output prompts include movement command tasks, visual picking tasks, and killing the current program tasks or stopping tasks:

[0045] For the movement command task, you need to issue an instruction to execute bashmove.sh; for the line clearing task, you need to issue an instruction to execute bash lidar.sh.

[0046] For the visual grasping task, you need to issue the bash color_track.sh command.

[0047] For killing or stopping the current program task, you need to issue the bash kill.sh command.

[0048] When a task is required to be executed, only the command needs to be given, such as when a moving task is requested, you only need to issue bash move.sh, without additional content (i.e. restrictions and taboos). According to different current usage scenarios, the corresponding prompt information is imparted to the large model in the question and answer session.

[0049] In an exemplary embodiment, the keyword matching module specifically comprises:

[0050] A keyword matching unit is configured to match keywords in the text information required by the user by using a combination of regular expressions, deep learning algorithms, and string search (Boyer-Moore, BM) algorithms, and determine keyword matching information.

[0051] Further, a variety of algorithms are used to redundantly implement English and Chinese keyword matching, including a combination of regular expression-based, deep learning method-based, and BM algorithm-based methods, which can extract English and Chinese keywords, and thus realize automatic running of ROS corresponding programs according to the output results of the large model.

[0052] In an exemplary embodiment, the ROS robot module specifically comprises:

[0053] An API interface information matching unit is configured to match API interface information of the ROS robot according to the keyword matching information.

[0054] A ROS program execution unit is configured to control the ROS robot to execute a corresponding ROS program according to the API interface information.

[0055] A dynamic modification unit is configured to dynamically modify parameter information of the ROS program through a ROS startup script, and assist in completing a teaching task.

[0056] In an exemplary embodiment, the ROS robot intelligent teaching auxiliary system based on the large model is connected to the ROS robot in the same local area network through a Secure SHell (SSH) protocol.

[0057] Further, the ROS robot provides an internal API interface according to an internal program algorithm, different programs of the ROS can be started through the API interface; the teaching system can obtain the corresponding API interface information according to the keyword matching information, and then start or stop the currently executing ROS program through the system; or the parameter information of the ROS program can be dynamically modified through the ROS start script, the teaching content of the teacher is enriched, and the physical meaning of the parameter and the engineering effect brought by the parameter are guided to the students through the on-site teaching mode.

[0058] Scenario 1: How to use the system to teach a PID control in a teaching classroom.

[0059] Prompt information: “You now have a ROS robot, which can move omnidirectionally and use a mechanical arm to pick up objects. The existing robot has the following programs: when performing corresponding tasks, you need to execute the corresponding program according to the ROS mechanism. If you need to issue a move.sh command to execute the bash move.sh instruction; if you need to issue a lidar.sh command to execute the bash lidar.sh instruction; if you need to issue a color_track.sh command to execute the bash color_track.sh instruction; if you need to issue a kill.sh command to execute the bash kill.sh instruction; when you need to execute a task, you only need to give an instruction, such as when you request a move task, you only need to issue bash move.sh, and no additional content is required.”

[0060] User demand 1: Please execute the move task.

[0061] Large model output: bash move.sh.

[0062] Keyword matching: according to the output result “bash move.sh” of the large model, the probability of matching the program for starting the robot movement in the ROS robot is the highest and reaches a threshold, and therefore the corresponding program command is automatically issued.

[0063] At this time, the program is normally started, and feedback is given to the system that the ROS robot is running the program.

[0064] At this time, the teacher knows that the current ROS movement program is developed according to the PID algorithm principle, and therefore can dynamically teach students how the three parameters P, I and D affect the control effect of the robot by modifying the values of the parameters P, I and D, and therefore can modify the corresponding parameters through the system after the move task is executed once.

[0065] User demand 2: Modify the parameter list in the move task, and modify the parameter p to 0.5.

[0066] Large model output: sed-i's / ^p=.* / p=0.5 / 'pid.txt.

[0067] Keyword matching: according to the large model result, if the similarity between the system preset command and the large model result is high and exceeds the threshold, the corresponding command is executed to modify the configuration file, and the system is fed back to inform that the modification is successful.

[0068] First, the user obtains voice through the microphone and converts the voice into text information through voice recognition technology, or inputs or changes the text information after voice recognition through the keyboard, then transmits the text information to the large model to obtain the question and answer result, performs keyword matching on the question and answer result, obtains the corresponding ROS program API through the matching result, connects the ROS robot through the local area network, executes the corresponding program to control the robot, and realizes the comprehensive integrated teaching of theory + code + engineering deployment + robot running test, as shown in Figure 2 .

[0069] Through the above process, the teacher does not need to input commands through the keyboard or manually search and modify parameters in the ROS robot's file system, which greatly saves the teacher's time. Through the system, the teacher's teaching content can be enriched, and the students can understand the physical meaning of the parameters and the engineering effect of modifying the parameters through on-site teaching.

[0070] The present application combines the large model with traditional teaching to improve the teacher's preparation efficiency, increase the interest and practicality of classroom teaching, and quickly realize the comprehensive integrated teaching of theory + code + engineering deployment + robot running test, which can assist the teacher to guide the students to quickly understand the teaching content from all aspects, thereby making up for the hollow limitations of traditional teaching in engineering practice.

[0071] The embodiment of the present application provides a ROS robot intelligent teaching auxiliary method based on a large model, which is executed by a computer device, specifically by a terminal or a server, or by a terminal and a server together. As shown in Figure 3 , the method comprises the following steps.

[0072] S1: according to the user's use scene, select an input mode, and input text information according to the input mode; the input mode includes a voice recognition text input mode and a keyboard input mode.

[0073] S2: based on a large model, according to the text information and large model prompt information, generate the text information required by the user; the large model prompt information includes theme, target, background information, role setting, input and output prompt, and restriction and taboo content.

[0074] S3: keyword matching information is determined according to the keyword matching of the required text information of the user.

[0075] S4: the ROS robot is controlled to execute the corresponding ROS program according to the keyword matching information, and the parameter information of the ROS program is dynamically modified to assist in completing the teaching task.

[0076] In an exemplary embodiment, a computer device is provided, Figure 4 As shown, the computer device can be a server or a terminal. The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control ability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the ROS robot intelligent teaching assistance data based on large models. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through network connection. The computer program is executed by the processor to realize a large model-based ROS robot intelligent teaching assistance method.

[0077] In an exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to realize the above method.

[0078] In an exemplary embodiment, a computer readable storage medium is provided, storing a computer program, which is executed by a processor to realize the above method.

[0079] In an exemplary embodiment, a computer program product is provided, including a computer program, which is executed by a processor to realize the above method.

[0080] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments of each method. Any reference to memory, databases or other media used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ReadOnly Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc.

[0081] In the present application, all actions of obtaining signals, information or data are carried out in accordance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization of the owner of the corresponding device.

[0082] The database involved in each embodiment provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in each embodiment provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0083] Each technical feature of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0084] The principles and implementation manners of the present application are described herein by using specific examples, and the above examples are only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will have changes. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A ROS robot intelligent teaching auxiliary system based on a large model, characterized by: include: A user input module is used to select an input mode according to the user's usage scenario and input text information according to the input mode; the input mode includes a voice recognition text input mode and a keyboard input mode; A large model module is used to generate text information required by the user based on the large model and the text information and large model prompt information; the large model prompt information includes the theme, goal, background information, role settings, input and output prompts, and restrictions and taboos; The large model module specifically includes: A matching unit, configured to dynamically match large model prompt information according to the text information; A user required text information generating unit, configured to input the large model prompt information into the large model to generate the user required text information; A keyword matching module is used to perform keyword matching based on the text information required by the user and determine keyword matching information; the keyword matching module specifically includes: A keyword matching unit, configured to perform keyword matching on the text information required by the user using a combination of regular expressions, deep learning algorithms, and BM algorithms to determine keyword matching information; The ROS robot module is used to control the ROS robot to execute the corresponding ROS program according to the keyword matching information, and dynamically modify the parameter information of the ROS program to assist in completing the teaching task; the ROS robot module specifically includes: An API interface information matching unit, configured to match the API interface information of the ROS robot according to the keyword matching information; A ROS program execution unit, configured to control the ROS robot to execute a corresponding ROS program according to the API interface information; The dynamic modification unit is used to dynamically modify the parameter information of the ROS program through the ROS startup script to assist in completing the teaching task.

2. The ROS robot intelligent teaching auxiliary system based on a large model according to claim 1 is characterized in that: The user input module specifically includes: A keyboard input unit, used for inputting text information using a keyboard input mode; A speech recognition unit, configured to recognize speech information and convert the speech information into text information using the speech recognition text input mode; The text information modification unit is used to modify the text information converted by the speech recognition unit by using a keyboard input mode.

3. The ROS robot intelligent teaching auxiliary system based on a large model according to claim 1 is characterized in that: The large-model-based ROS robot intelligent teaching auxiliary system and the ROS robot are in the same local area network and are connected via SSH.

4. A ROS robot intelligent teaching auxiliary method based on a large model, characterized in that: The ROS robot intelligent teaching assistance method based on a large model is applied to the ROS robot intelligent teaching assistance system based on a large model according to any one of claims 1 to 3, and the ROS robot intelligent teaching assistance method based on a large model includes: Select an input mode according to the user's usage scenario, and input text information according to the input mode; the input mode includes voice recognition text input mode and keyboard input mode; Based on the big model, generating the text information required by the user according to the text information and the big model prompt information; the big model prompt information includes the theme, goal, background information, role settings, input and output prompts, and restrictions and taboos; Perform keyword matching based on the text information required by the user to determine keyword matching information; According to the keyword matching information, the ROS robot is controlled to execute the corresponding ROS program, and the parameter information of the ROS program is dynamically modified to assist in completing the teaching task.

5. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the large-model-based ROS robot intelligent teaching assistance method according to claim 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the ROS robot intelligent teaching auxiliary method based on a large model as described in claim 4 is implemented.

7. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the ROS robot intelligent teaching auxiliary method based on a large model as described in claim 4 is implemented.

Images