An intelligent AI research and teaching system and method in a cooking classroom scenario

Abstract

The present invention discloses an intelligent AI research and training teaching system in a cooking classroom scenario, comprising an edge-side video terminal and a remote server end; the edge-side video terminal comprises a teacher end and several student ends; the remote video terminal comprises a course database and an AI research and training platform; the course database comprises several cooking process data, and the AI ​​research and training platform performs real-time monitoring of audio and video data and sensor data transmitted by the edge-side video terminal based on the cooking process data; the present invention also discloses an AI research and training teaching method, which provides quantitative instruction commands for the teacher's demonstration teaching behavior through real-time analysis by the remote server end while recording and broadcasting the teacher's demonstration teaching video in real time, and uses the AI ​​model to compare the actual operation behavior in real time, and issues reminder instructions for improper operation behavior.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent research and training technology, and in particular relates to an intelligent AI research and training teaching system and method in a cooking classroom scenario. Background Art

[0002] At present, various talent training models such as vocational education and skills training are developing at a high speed, and the teaching and training content based on various segmented scenarios has different scenario characteristics. At present, there are common problems in different vocational education scenarios, namely the problem of onlooker teaching. The teaching and training scenes lack a sense of reality, the training conditions are not perfect, and it is difficult for students to master professional skills in a real working environment. The number of training equipment is limited, and it is impossible to meet the needs of all students to operate the training at the same time. The onlooker teaching causes many students to be unable to clearly watch the teacher's training process. Specifically in the cooking classroom scenario, due to the real-time nature of the training, the teacher cannot guide each student in real time. In summary, solving the problem of onlooker teaching and providing real-time guidance for students' operations during cooking education are urgent issues to be solved. Summary of the invention

[0003] Purpose of the invention: In view of the problems existing in the above-mentioned background technology, the present invention provides an intelligent AI training and teaching system and method in a cooking classroom scenario. Based on the traditional multi-terminal recording and broadcasting, an AI training and teaching method is provided to solve the problem that teachers cannot provide real-time guidance in the practical cooking process. The method analyzes and judges the students' practical practice process based on image and sensor information, and provides real-time reminders, which effectively solves the problem that practical cooking cannot be taught in a timely manner.

[0004] Technical solution: An intelligent AI research and training teaching system in a cooking classroom scenario, including an edge-side video terminal and a remote server; the edge-side video terminal includes a teacher terminal and several student terminals; each of the edge-side video terminals includes an edge-side computing terminal, a camera, an array microphone, an IoT sensor module and a display module; the remote server includes a course database and an AI research and training platform; the course database includes several cooking process data, and the AI ​​research and training platform monitors the audio and video data and sensor data transmitted by the edge-side video terminal in real time based on the cooking process data, conducts comprehensive analysis, and gives standardized instruction commands and corresponding guidance methods.

[0005] Furthermore, the IoT sensor module includes an infrared sensor for measuring ambient temperature and a gravity sensor for measuring the weight of additives; the edge video terminal is provided with a tray for weighing, and the gravity sensor is integrated inside the tray. The edge computing terminal receives and processes data from the camera, array microphone and IoT sensor module, communicates with the remote server, and controls the display content of the display module.

[0006] Furthermore, the edge-side video terminal collects the cooking operation video stream in real time, and transmits it to the AI ​​research and training teaching platform synchronously with the sensor data of the IoT sensor module; the teacher end receives the practical video images transmitted by each student end in real time, and the student end synchronously receives the demonstration teaching images transmitted by the teacher end.

[0007] An intelligent AI research and teaching method in a cooking classroom scenario includes the following steps:

[0008] Step S1: For the current course teaching dishes, select standard cooking process data from the course database and load it to the AI ​​research and training teaching platform; the remote server enters the class preparation;

[0009] Step S2: When the teacher conducts a demonstration teaching, the teacher terminal transmits the audio and video data of the operation process and the corresponding IoT sensor data to the AI ​​research and teaching platform in real time; the AI ​​research and teaching platform performs feature recognition on the teacher's operation video, and combines audio data recognition and sensor data analysis to generate a number of instruction commands based on the operation content, and synchronously transmits the instruction commands and the demonstration teaching screen to the student terminal for demonstration teaching;

[0010] Step S3: When students are practicing cooking, the video images collected by the student end are transmitted to the teacher end and the remote server end respectively, and the various sensor data are transmitted to the AI ​​research and training teaching platform at the same time; the AI ​​research and training teaching platform monitors the cooking practice of the student end based on the instruction command; when an improper operation occurs on a certain student end, the AI ​​research and training teaching platform sends a reminder command to the corresponding student end;

[0011] Step S4: When a student terminal of a certain route sends out m prompt instructions in a continuous n-segment instruction command, the AI ​​training and teaching platform notifies the teacher terminal, and the teacher focuses on the corresponding students according to the teacher terminal's reminder instructions, and provides on-site guidance when necessary;

[0012] Step S5: After the teaching is completed, the teacher will upload the standard cooking operation video stream and instruction commands to the remote server synchronously for students to review after class.

[0013] Furthermore, in step S2, the AI ​​training and teaching platform collects various process data during the teacher's actual operation, corresponds to the cooking process data obtained in step S1, and generates corresponding instruction commands; the instruction commands include temperature data, type of additives, weight of additives and interval time; the interval time represents the interval time between two addition processes; repeat the operation and obtain instruction commands until the cooking operation is completed; after the practical operation is completed, the teacher's end locally stores the current practical demonstration video and the generated instruction commands, and at the same time transmits the generated instruction commands to the student's end display module for reference in the student's practical exercises.

[0014] Furthermore, in each instruction command generated, the temperature data, additive weight and interval duration all include guidance values ​​and guidance intervals; the guidance values ​​are standard values ​​used by teachers during demonstration teaching, and the guidance intervals are guidance intervals given in the corresponding cooking processes; the student-end display module simultaneously displays the guidance values ​​and guidance intervals.

[0015] Furthermore, in step S3, the specific method for determining whether the student terminal has improper operation includes:

[0016] (1) At the beginning of each instruction command, the current temperature in the pot is obtained. When the temperature in the pot reaches the upper limit of the guidance interval corresponding to the temperature data and no additives are detected within th1 seconds, a reminder command is issued; th1 is the time threshold;

[0017] (2) Identify the type of current additives through images, and comprehensively judge the type and weight of the additives based on the student's weighing situation; when the type of additives does not match the instruction command, or the absolute value of the difference between the additive weight and the upper limit of the additive weight guidance range in the instruction command is greater than the weight threshold th2, issue a reminder instruction;

[0018] (3) When the interval between two additions is less than or equal to the lower limit of the interval guidance range, it means that the addition is too early, and a reminder instruction is issued.

[0019] Compared with the prior art solutions, the technical solution adopted by the present invention has the following beneficial effects:

[0020] In view of the problem of students watching the teaching in a bystander manner during the current cooking teaching process, the present invention provides a multi-terminal real-time recording and broadcasting teaching system based on the student end, the teacher end and the remote server end. While the teacher's demonstration teaching video is recorded and broadcasted in real time, the remote server end conducts real-time analysis and corresponds to the cooking process data to provide quantitative instruction commands for the teacher's demonstration teaching behavior for students' reference and remote analysis and detection.

[0021] In response to the problem that teachers are unable to provide real-time guidance during students' practical operations, the present invention provides a research, training and teaching method based on AI analysis. It collects sensor and video information and conducts real-time comparison and monitoring with quantitative instruction commands. When improper operations occur, reminder instructions are issued in real time. For students with prominent problems, teachers are reminded to provide on-site guidance, thereby improving teaching quality and guidance efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of the intelligent AI research and training teaching system in a cooking classroom scenario provided by the present invention;

[0023] Figure 2 A schematic diagram of an edge-side video terminal system in a cooking classroom scenario provided by the present invention;

[0024] Figure 3 This is a schematic diagram of the intelligent AI training and teaching method in a cooking classroom scenario provided by the present invention. DETAILED DESCRIPTION

[0025] The present invention provides an intelligent AI training and teaching system and method in a cooking classroom scenario. It provides a comprehensive training and teaching system based on IoT sensor information for the cooking classroom scenario. On the basis of the traditional real-time recording and broadcasting function, it performs intelligent analysis of key cooking steps through image analysis, and monitors important information in the cooking process in real time through sensors, such as temperature, weight of additives, etc., to monitor and guide students' cooking practice in real time. The principles of the intelligent AI training and teaching system and method provided by the present invention are explained in detail below in conjunction with the drawings of the specification.

[0026] like Figure 1-Figure 2 As shown, the AI ​​research and training teaching system provided by the present invention includes an edge-side video terminal and a remote server. The edge-side video terminal includes a student terminal and a teacher terminal; each video terminal includes an edge-side computing terminal for data processing and transmission, a camera for video acquisition, an array microphone, an IoT sensor module and a display module for collecting scene information. The IoT sensor module includes a gravity sensor for real-time weighing, an infrared sensor for real-time temperature measurement and other sensor devices. The student terminal transmits the collected video stream data and various sensor device data to the teacher terminal and the remote server terminal in real time. The teacher terminal synchronously transmits the collected video stream data and various IoT sensor data to the student terminal and the remote server terminal. The remote server terminal includes a course database and an AI research and training platform. The course database includes a number of cooking process data; the cooking process data includes various data such as dish type, dish weight, seasoning type, seasoning weight, pot entry order, oil temperature standard, etc., which are used to describe the standard cooking process of a dish. Based on cooking process data, the AI ​​research and training platform monitors the audio and video data and sensor data transmitted by the edge-side video terminal in real time, conducts comprehensive analysis, and gives standardized instruction commands and corresponding guidance methods.

[0027] The AI ​​research and training teaching method provided by the present invention is as follows Figure 3 As shown, specifically including:

[0028] Step S1: For the dishes taught in the current course, standard cooking process data is selected from the course database, and the remote server enters the class preparation.

[0029] The teacher selects standard cooking process data according to the current course requirements, makes adaptive modifications, and inputs the modified cooking process data into the AI ​​research and training teaching platform. The AI ​​research and training teaching platform generates several instruction commands based on the cooking process data to guide students to complete cooking operations step by step. At the same time, a real-time monitoring algorithm is embedded in the AI ​​research and training teaching platform to monitor whether there is improper operation on each student end.

[0030] Step S2: During the cooking demonstration, the teacher collects the teacher's operation video stream and IoT sensor data in real time, and transmits them to the student and remote server simultaneously. The remote server recognizes the characteristics of the teacher's operation video through the AI ​​training and demonstration platform, analyzes the audio data and sensor data, and corresponds the teacher's operation steps to the instruction commands described in step S1, and transmits the instruction commands to the student in real time according to the teacher's operation sequence.

[0031] When the teacher performs the practical demonstration teaching, he / she performs standard cooking operations according to each dish. At this time, the AI ​​training and teaching system performs image recognition based on the video frame of the video stream transmitted in real time by the teacher, and generates corresponding instruction commands based on the sensor data and the collected audio data. Each instruction command includes temperature data, type of additives, weight of additives, and interval time. The interval time represents the interval time between two addition processes.

[0032] When the sensor monitors the temperature in the pot in real time, when the image recognizes the action of adding oil, or recognizes the keyword "add oil" appearing in the teacher's audio data, the AI ​​research and training teaching platform retrieves the corresponding cooking process data, and extracts the instruction commands related to the temperature in the pot and the amount of oil, and generates the instruction command "The temperature in the pot is a degree, add cooking oil, and add weight b grams". Continue to monitor the oil temperature in the pot until the next step of adding other ingredients, calculate the interval time, and generate a complete instruction command. It should be noted that under normal circumstances, the weight of the additives is obtained by weighing with a gravity sensor to obtain a standard value. If the teacher does not weigh during standard operation, the AI ​​research and training teaching platform will give the weight range of the additives according to the corresponding cooking process data.

[0033] When the teacher continues to add other ingredients, the AI ​​research and training teaching platform recognizes the specific ingredients through images, retrieves the cooking process data, and repeats the generation process of the above instruction commands until the cooking operation is completed. After the practice is completed, the teacher's end locally stores the current practical demonstration video and the generated instruction commands, and transmits the generated instruction commands to the student-end display module for reference for the student's practical practice. For each instruction command generated in this embodiment, its temperature data, additive weight and interval duration include guidance values ​​and guidance intervals; the guidance values ​​are standard values ​​for teachers to demonstrate teaching, and are used for students to accurately refer to. The guidance interval is the guidance interval given in the corresponding cooking process, which represents the controllable range of various indicators during student practice, and is used for subsequent AI research and training teaching platform to conduct real-time detection of student practice. The student-end display module displays the guidance value and guidance interval at the same time.

[0034] Step S3: When students are performing practical operations, the collected video images are transmitted to the teacher and the remote server, and the real-time monitored sensor data is transmitted to the remote server. The AI ​​research and training teaching platform compares the video data transmitted by each student terminal, the real-time transmission data of each sensor, and the instruction command in real time. When an improper operation occurs on a student terminal, the AI ​​research and training teaching platform sends a reminder command to the corresponding student terminal.

[0035] The specific method for judging improper operation of the student terminal in the present invention includes:

[0036] (1) At the beginning of each instruction command, the current temperature in the pot is obtained. When the temperature in the pot reaches the upper limit of the guidance interval corresponding to the temperature data and no additives are detected within th1 seconds, a reminder command is issued; th1 is the time threshold;

[0037] (2) Identify the type of current additives through images, and comprehensively judge the type and weight of the additives based on the student's weighing situation; when the type of additives does not match the instruction command, or the absolute value of the difference between the additive weight and the upper limit of the additive weight guidance range in the instruction command is greater than the weight threshold th2, issue a reminder instruction;

[0038] (3) When the interval between two additions is less than or equal to the lower limit of the interval guidance range, it means that the addition is too early, and a reminder instruction is issued.

[0039] Step S4: When a student terminal sends a total of m prompt instructions in n consecutive instruction commands, the AI ​​training and teaching platform notifies the teacher terminal. The teacher focuses on the corresponding students according to the teacher terminal's reminder instructions and provides on-site guidance when necessary.

[0040] Step S5: After the teaching is completed, the teacher will upload the standard cooking operation video stream and instruction commands to the remote server synchronously for students to review after class.

[0041] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An intelligent AI research and training teaching method in a cooking classroom scenario, applied to an intelligent AI research and training teaching system in a cooking classroom scenario, including an edge-side video terminal and a remote server; the edge-side video terminal includes a teacher terminal and several student terminals; each of the edge-side video terminals includes an edge-side computing terminal, a camera, an array microphone, an IoT sensor module and a display module; the remote server includes a course database and an AI research and training platform; the course database includes several cooking process data, and the AI ​​research and training platform monitors the audio and video data and sensor data transmitted by the edge-side video terminal in real time based on the cooking process data, conducts comprehensive analysis, and gives standardized instruction commands and corresponding Guidance method; the IoT sensor module includes an infrared sensor for measuring ambient temperature and a gravity sensor for measuring the weight of additives; the edge-side video terminal is provided with a tray for weighing, and the gravity sensor is integrated inside the tray; the edge-side computing terminal receives and processes data from the camera, array microphone and IoT sensor module, communicates with the remote server, and controls the display content of the display module; the edge-side video terminal collects the cooking operation video stream in real time, and transmits it synchronously with the sensor data of the IoT sensor module to the AI ​​research and training teaching platform; the teacher end receives the practical video screen transmitted by each student end in real time, and the student end synchronously receives the demonstration teaching screen transmitted by the teacher end; it is characterized in that The following steps are involved: Step S1: For the current course teaching dishes, select standard cooking process data from the course database and load it to the AI ​​research and training teaching platform; The remote server enters the class preparation; Step S2: When the teacher conducts a demonstration teaching, the teacher terminal transmits the audio and video data of the operation process and the corresponding IoT sensor data to the AI ​​research and teaching platform in real time; the AI ​​research and teaching platform performs feature recognition on the teacher's operation video, and combines audio data recognition and sensor data analysis to generate a number of instruction commands based on the operation content, and synchronously transmits the instruction commands and the demonstration teaching screen to the student terminal for demonstration teaching; Step S3: When students are practicing cooking, the video images collected by the student end are transmitted to the teacher end and the remote server end respectively, and the various sensor data are transmitted to the AI ​​research and training teaching platform at the same time; the AI ​​research and training teaching platform monitors the cooking practice of the student end based on the instruction command; when an improper operation occurs on a certain student end, the AI ​​research and training teaching platform sends a reminder command to the corresponding student end; Step S4: When a student terminal of a certain route sends out m prompt instructions in a continuous n-segment instruction command, the AI ​​training and teaching platform notifies the teacher terminal, and the teacher focuses on the corresponding students according to the teacher terminal's reminder instructions, and provides on-site guidance when necessary; Step S5: After the teaching is completed, the teacher will upload the standard cooking operation video stream and instruction commands to the remote server synchronously for students to review after class.

2. According to claim 1, the intelligent AI training and teaching method in a cooking classroom scenario is characterized in that: In step S2, the AI ​​training and teaching platform collects various process data during the teacher's actual operation, corresponds to the cooking process data obtained in step S1, and generates corresponding instruction commands; the instruction commands include temperature data, type of additives, weight of additives and interval duration; the interval duration represents the interval between two addition processes; repeat the operation and obtain instruction commands until the cooking operation is completed; after the actual operation is completed, the teacher's end locally stores the current practical demonstration video and the generated instruction commands, and at the same time transmits the generated instruction commands to the student's end display module for reference in the student's practical exercises.

3. The intelligent AI training and teaching method in a cooking classroom scenario according to claim 2 is characterized in that: In each instruction command generated, the temperature data, the weight of the additives and the interval duration all include guidance values ​​and guidance intervals; the guidance values ​​are standard values ​​used by teachers for demonstration teaching, and the guidance intervals are guidance intervals given in the corresponding cooking processes; the student-end display module simultaneously displays the guidance values ​​and guidance intervals.

4. The intelligent AI training and teaching method in a cooking classroom scenario according to claim 3 is characterized in that: In step S3, the specific method for determining improper operation of the student terminal includes: (1) At the beginning of each instruction command, the current temperature in the pot is obtained; when the temperature in the pot reaches the upper limit of the guidance interval corresponding to the temperature data and no additives are detected within th1 seconds, a reminder command is issued; where th1 is the time threshold; (2) Identify the type of current additives through images, and comprehensively judge the type and weight of the additives based on the student's weighing situation; when the type of additives does not match the instruction command, or the absolute value of the difference between the additive weight and the upper limit of the additive weight guidance range in the instruction command is greater than the weight threshold th2, issue a reminder instruction; (3) When the interval between two additions is less than or equal to the lower limit of the interval guidance range, it means that the addition is too early, and a reminder instruction is issued.

Images