A graphical creative programming support method and system for classroom environment

Abstract

The present invention discloses a graphical creative programming support method for a classroom environment, comprising the following steps: obtaining user interaction information through a preset large language model; extracting keywords from the interaction information according to the content involved in the teaching plan and encoding them through the large language model to form a triple set; obtaining a standard code set, and converting the image format into a pseudo code in text form through the large language model, and forming a pseudo code set by combining the standard code set and the pseudo code; using the teaching knowledge and code programming logic in the triple set as nodes and the relationships as edges to construct a corresponding semantic relationship network; using the triple set and the pseudo code set to train the semantic relationship network to obtain a generation model. The present invention also provides a graphical creative programming support system. The method provided by the present invention guides users to understand and learn programming logic in an interactive dialogue question-and-answer format, thereby improving users' classroom enthusiasm.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence-assisted teaching, and in particular relates to a graphical creative programming support method and system for use in a classroom environment. Background Art

[0002] Graphical programming: This method uses a graphical interface to create and connect program components. Instead of writing traditional code, users drag and drop graphical blocks or elements to implement program functions. This method is particularly suitable for beginners and children because it lowers the barrier to entry and allows users to intuitively understand program structure and logic.

[0003] Project-Based Learning: Project-Based Learning (PBL) is a student-centered teaching approach that encourages students to explore and respond to real-world problems or challenges through hands-on projects. This approach emphasizes deep learning, and students must not only master subject knowledge in the process of completing projects, but also develop key thinking skills such as critical thinking, creativity, collaboration, and communication. In project-based learning, the teacher's role is more as a guide and facilitator rather than a traditional knowledge transmitter. Students usually work within a guided framework that defines the project's goals, deadlines, and expected outcomes. Projects are often interdisciplinary, combining content and skills from multiple subject areas to solve complex problems or create innovative products.

[0004] Large language models: These are deep learning models trained using large amounts of text data. These models are capable of understanding and generating human language. These models are characterized by numerous model parameters and extensive training data, enabling them to handle complex language tasks such as text generation, translation, summarization, and question-answering.

[0005] Text-to-image models use artificial intelligence, particularly deep learning, to generate images based on textual descriptions. These models are able to understand the descriptions in text and create visual images that match the descriptions, a process that often involves sophisticated computer vision and natural language processing techniques.

[0006] Text-to-audio models use artificial intelligence (AI), particularly deep learning, to generate audio output based on text input. These audio outputs can be speech, music, or other sound effects. These models typically involve complex natural language processing and audio processing techniques to convert text content into audible audio.

[0007] Patent document CN117608555A discloses a programming assistance method and system that supports creativity and graphics. The method includes the following steps: obtaining sketch data and voice data describing the sketch data to construct a corresponding image material set; recording the voice instructions for programming the sketch data as instruction voice data to generate corresponding code blocks; selecting standard image materials from a material library based on the programming requirements, entering the corresponding voice instructions to generate visual code, and editing and combining the generated code blocks to output code text for implementing programming operations on the standard image materials. This method can only retrieve and generate programming information based on user input, and cannot highlight key content to increase user enthusiasm for understanding programming logic. Summary of the Invention

[0008] The purpose of the present invention is to provide a graphical creative programming support method and system for a classroom environment, which guides users to understand and learn programming logic in an interactive dialogue question-and-answer format, thereby improving users' classroom enthusiasm.

[0009] In order to achieve the first object of the present invention, the following technical solution is provided: a graphical creative programming support method for a classroom environment, comprising the following steps:

[0010] Obtaining user interaction information through a preset large language model, which includes the prompt word text preset by the large language model, the interaction text input by the user into the large language model, and the context text generated during the interaction between the user and the large language model;

[0011] Extract keywords from interactive information based on the content of the teaching plan and encode them through a large language model to generate a set of triples consisting of teaching knowledge, relationships, and code programming logic;

[0012] Obtain a standard code set containing multiple graphical code blocks and corresponding programming requirements, and convert the image format into textual pseudocode using a large language model, and combine the programming requirements corresponding to the graphical code blocks and the pseudocode to form a pseudocode set;

[0013] Using the teaching knowledge and code programming logic in the triple set as nodes and the relationships as edges, a corresponding semantic relationship network is constructed;

[0014] The semantic relationship network is trained using the triple set and the pseudocode set to obtain a generation model for standard code generation, wherein the generation model includes a logical keyword extraction module, a mapping module, a screening module, and a prediction module;

[0015] The logic keyword extraction module is used to extract logic keywords from input programming requirements;

[0016] The mapping module is used to search the interactive information according to the input logical keywords, connect nodes in the semantic network according to the search results, and output the corresponding triple set;

[0017] The screening module is used to screen the triplet set output by the mapping module according to the teaching knowledge and programming requirements in the teaching plan, and retain the pseudocode related to the teaching knowledge;

[0018] The prediction module predicts the corresponding graphical code block based on the pseudo code retained by the screening module;

[0019] The user's coding requirements are input into the generation model corresponding to the teaching plan to output graphical code blocks that conform to the teaching plan content.

[0020] The present invention constructs a relationship network between user interaction information and logical keywords, and uses a multimodal method to obtain the user's coding requirements. The relationship network generates corresponding graphical code blocks, and the obtained graphical code blocks are used to output the next step of voice, text or image.

[0021] Specifically, the graphical code block is constructed based on the block palette of Scratch.

[0022] Specifically, a saliency filter is used to associate teaching knowledge with code programming logic to guide the large language model to determine which code blocks should be highlighted and classified as high or low importance, thereby reducing students' confusion during the learning process.

[0023] Specifically, the expression of the saliency filter is as follows: ;in, Indicates the A graphical code block, Indicates the The graphical code blocks are closely related to the teaching knowledge. Represents a collection of graphical code blocks.

[0024] Specifically, the prediction module uses a normalized Levenshtein edit distance algorithm to predict between the pseudocode and the graphical code block.

[0025] Specifically, the expression of the Levenshtein edit distance algorithm is as follows: ; ; Among them, x, y represent the generated code block and the real code block, The ID of the matched code block.

[0026] Specifically, the generation model also includes a prediction output module, which includes image output and voice output. The image output is based on generating a corresponding graphical code block through a Canny edge detector, and the voice output uses a stable diffusion algorithm to convert the text of the graphical code block into audio.

[0027] To achieve the second objective of the present invention, the following technical solution is provided: a graphical creative programming support system, implemented by the above-mentioned graphical creative programming support method for a classroom environment, comprising:

[0028] An input module, used for inputting data, including voice data, text data and image data;

[0029] An interaction module, used to interact with the user based on input data to obtain interaction information;

[0030] A database module for storing pre-built standard code collections;

[0031] A data analysis module is used to analyze the interactive information to obtain the corresponding graphical code block from the database module;

[0032] The result output module is used to output the obtained graphical code blocks in one or more forms of images, text or audio.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The user's interaction information is obtained from the interactive process, and the corresponding graphical code blocks are generated using the generative model built based on the teaching plan. By highlighting the code blocks closely related to the educational objectives, it ensures that students' programming projects can better reflect the educational objectives. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A flowchart of the graphical creative programming support method for a classroom environment provided by this embodiment;

[0036] Figure 2 A flow chart of the training and interaction of the generative model provided in this embodiment;

[0037] Figure 3 This is a schematic diagram of the interface of the graphical creative programming support system provided in this embodiment. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0039] Since the traditional programming teaching process is relatively boring, how to improve classroom efficiency, help teachers complete teaching objectives with high quality when facing a large number of students, and ensure that every student can give full play to their creativity while completing programming learning goals has become a technical problem to be solved by this invention.

[0040] like Figure 1 As shown, the graphical creative programming support method provided by this embodiment for use in a classroom environment assists students in learning through step-by-step logic expansion and code generation, including the following process:

[0041] Obtaining user interaction information through a preset large language model, which includes the prompt word text preset by the large language model, the interaction text input by the user into the large language model, and the context text generated during the interaction between the user and the large language model;

[0042] Extract keywords from interactive information based on the content of the teaching plan and encode them through a large language model to generate a set of triples consisting of teaching knowledge, relationships, and code programming logic;

[0043] Obtain multiple graphical code blocks and corresponding programming requirements, convert the image format into textual pseudocode using a large language model, and combine the programming requirements and pseudocode corresponding to the graphical code blocks into a pseudocode set;

[0044] Using the teaching knowledge and code programming logic in the triple set as nodes and the relationships as edges, a corresponding semantic relationship network is constructed;

[0045] The semantic relationship network is trained using the triple set and the pseudocode set to obtain a generation model for generating graphical code blocks, wherein the generation model includes a logical keyword extraction module, a mapping module, a screening module, and a prediction module;

[0046] The logic keyword extraction module is used to extract logic keywords from input programming requirements;

[0047] The mapping module is used to search the interactive information according to the input logical keywords, connect nodes in the semantic network according to the search results, and output the corresponding triple set;

[0048] The screening module is used to screen the triplet set output by the mapping module according to the teaching knowledge and programming requirements in the teaching plan, and retain the pseudocode related to the teaching knowledge;

[0049] The prediction module predicts the corresponding graphical code block based on the pseudo code retained by the screening module;

[0050] The user's coding requirements are input into the generation model corresponding to the teaching plan to output graphical code blocks that conform to the teaching plan content.

[0051] More specifically, this embodiment adopts a specific mind map structure, which organizes information through nodes distinguished by three colors, representing different programming elements: programming roles (green), programming logic (purple) and code (gray).

[0052] It also integrates a Scratch-based block palette that allows students to intuitively select and add programming blocks, thus simplifying the programming learning process.

[0053] The method provided in this embodiment uses an additional large language model to automatically annotate the content generated by the large language model, and automatically adds these contents as nodes to the mind map. This process not only enriches the semantic information of the mind map, but also helps students understand programming knowledge more deeply by graphically displaying complex relationships.

[0054] At the same time, in order to make the model more suitable for teaching tasks, teachers can set the initial mind map template and predefined learning objectives before class, and use these objectives to control the large language model to generate content, ensuring that all added nodes are related to the learning objectives. The large language model is used to determine whether the generated nodes are key project nodes, and these key nodes are highlighted to guide students.

[0055] For the aforementioned large language model, an interactive Q&A chat box is used to encourage students to take an active role in the classroom, rather than waiting for the teacher's assistance. This example uses three stages of questions to guide students, asking questions that guide them to think about the roles or actions they need. For example, "What roles will your project include?" Students can choose to manually add nodes or click the "Generate Roles" button. They can then select a generated role and add it to their mind map.

[0056] To address the problem of prompt words for large language models, this embodiment uses specialized prompts for different stages, avoiding the performance degradation problem caused by merging all instructions into a single prompt in traditional methods. The process covers three main tasks consistent with the classroom processing process: (1) formulating questions to guide students to decompose tasks, involving conceptualizing roles, actions, and events; (2) formulating questions to assist students in creating materials, including drawing and audio generation; (3) providing programming logic assistance or automatically generating code based on student input. At the same time, to ensure the consistency of LLM generated content, user-created mind maps are used as the data basis for prompting LLM at each stage, thereby improving the accuracy and relevance of prompts.

[0057] like Figure 2 As shown, this embodiment enhances the interpretability of generated content through the use of a large language model (LLM). A small number of sample hints are used to guide an additional, small-parameter large language model to generate semantic relationships between nodes, expressed as triples such as <topic, relation, role> and <logic, relation, code>. To prevent information overload, only the edges connecting adjacent nodes generated by the LLM are annotated. Code blocks closely related to the educational objectives are highlighted to ensure that students' programming projects better reflect the educational objectives.

[0058] At the same time, a saliency filter is used to manage the complexity of the mind map and guide the large language model to determine which code blocks should be highlighted and classified as high or low importance, thereby reducing students' confusion during the learning process. The use process of the saliency filter is as follows: Step 1: Enter a small number of sample prompts;

[0059] Defining a small number of sample hints , guiding the small-parameter large language model to generate semantic relations between nodes.

[0060] .

[0061] Step 2: Generate triples; use a small number of samples to prompt the generation of triples : ;in, is a collection of nodes, Indicates the A triplet.

[0062] Step 3: Label the edges connecting adjacent nodes; only label the edges connecting adjacent nodes generated by the large language model : ;in, Representation node and are adjacent.

[0063] Step 4: Highlight code blocks; identify and align with educational objectives A collection of closely related code blocks : ;in, is the collection of all code blocks, Represents a code block and teaching objectives Closely related.

[0064] Step 5: Saliency Filtering; Use saliency filters to manage the complexity of the mind map and categorize the importance of code blocks and : ;in, Represents a code block importance.

[0065] The prediction module uses the normalized Levenshtein edit distance algorithm to predict between pseudocode and graphical code blocks. It passes through all pseudocode blocks in the AST generated by LLM, then calculates the edit distance with all class IDs, and designates the class ID with the smallest distance as the generated block ID, thereby visualizing the code blocks required by students and helping them correspond to Scratch more easily.

[0066] The generative model also includes a predictive output module, which includes image and speech output. The image output is generated based on the corresponding graphical code blocks using a Canny edge detector, while the speech output uses a stable diffusion algorithm to convert the text of the graphical code blocks into audio. Specifically, a large language model is used to convert student input into prompts that match the image and audio generative models. A role-playing strategy ensures that the large language model plays an appropriate educational and supportive role.

[0067] This embodiment further provides a graphical creative programming support system, which is implemented by the graphical creative programming support method for a classroom environment provided in the above embodiment, including:

[0068] An input module, used for inputting data, including voice data, text data and image data;

[0069] An interaction module, used to interact with the user based on input data to obtain interaction information;

[0070] A database module for storing pre-built standard code collections;

[0071] A data analysis module is used to analyze the interactive information to obtain the corresponding graphical code block from the database module;

[0072] The result output module is used to output the obtained graphical code blocks in one or more forms of images, text or audio.

[0073] In order to better illustrate the technical effects provided by this application, the following Figure 3 The interface diagram of the graphical creative programming support system is shown.

[0074] in, Figure 3 There is a block palette that contains all the graphical code blocks related to the teaching plan.

[0075] Figure 3 The node corresponding to a1 in represents the student, Figure 3 The node corresponding to a3 in represents the teacher. Figure 3 The a2 corresponding node in represents the AI in the large language model.

[0076] The three types of nodes are marked by connecting edges, such as Figure 3 As shown in a5 and a6, the highlighted nodes indicate their relevance to the learning objectives.

[0077] At the same time, through the question-answer mechanism Figure 3 Structured guidance and real-time support as shown in b1 and b2, while Figure 3 a7 in the text box indicates that the text input box allows audio material to be generated through text editing.

[0078] For example, students can choose to manually add nodes or click the "Generate Character" button. They can then select a generated character and add it to their mind map. During the material creation phase, if a student clicks a green-marked character node, the system will ask, "Would you like to add a picture or voice for this fun character?" A chat box allows students to switch to an AI-powered drawing board or pop up a text-to-audio input box, facilitating material creation. During the code implementation phase, a large language model encourages students to elaborate on their programming ideas. Students can get help by clicking the "Generate Logic" or "Generate Code" buttons in the chat box. For example, if a child provides a brief description of a character without many details, the agent will ask a follow-up question: "Great! Would you like to add an action for this character?" If the child is still confused, they can click the "Generate Logic" button in the chat window, and the system will generate various programming logic options for them to choose from.

[0079] In summary, this application obtains user interaction information from the interactive process, and uses a generation model built based on the teaching plan to generate corresponding graphical code blocks, and highlights the code blocks closely related to the educational objectives to ensure that students' programming projects can better reflect the educational objectives.

[0080] In addition, the terms "upper", "lower", "inner", "outer", "front", and "back" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. Unless otherwise specifically stated, the relative steps, numerical expressions, and values of the components and steps described in these embodiments do not limit the scope of the present invention.

[0081] Of course, the above description is only a specific embodiment of the present invention and is not intended to limit the scope of implementation of the present invention. Any equivalent changes or modifications made based on the structure, features and principles described in the scope of the patent application of the present invention should be included in the scope of the patent application of the present invention.

[0082] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A graphical creative programming support method for a classroom environment, characterized in that: The following steps are involved: Obtaining user interaction information through a preset large language model, which includes the prompt word text preset by the large language model, the interaction text input by the user into the large language model, and the context text generated during the interaction between the user and the large language model; Extract keywords from interactive information based on the content of the teaching plan and encode them through a large language model to generate a set of triples consisting of teaching knowledge, relationships, and code programming logic; Obtain multiple graphical code blocks and corresponding programming requirements, convert the image format into textual pseudocode using a large language model, and combine the programming requirements and pseudocode corresponding to the graphical code blocks into a pseudocode set; Using the teaching knowledge and code programming logic in the triple set as nodes and the relationships as edges, a corresponding semantic relationship network is constructed; The semantic relationship network is trained using the triple set and the pseudocode set to obtain a generation model for generating graphical code blocks, wherein the generation model includes a logical keyword extraction module, a mapping module, a screening module, and a prediction module; The logic keyword extraction module is used to extract logic keywords from input programming requirements; The mapping module is used to search the interactive information according to the input logical keywords, connect nodes in the semantic network according to the search results, and output the corresponding triple set; The screening module is used to screen the triplet set output by the mapping module according to the teaching knowledge and programming requirements in the teaching plan, and retain the pseudocode related to the teaching knowledge; The prediction module predicts the corresponding graphical code block based on the pseudo code retained by the screening module; The user's coding requirements are input into the generation model corresponding to the teaching plan to output graphical code blocks that conform to the teaching plan content.

2. The graphical creative programming support method for classroom environment according to claim 1, characterized in that: The graphical code blocks are constructed based on the block palette of Scratch.

3. The graphical creative programming support method for classroom environment according to claim 1, characterized in that: A saliency filter is used to associate teaching knowledge with code programming logic.

4. The graphical creative programming support method for classroom environment according to claim 3, characterized in that: The expression of the saliency filter is as follows: Among them, importance(c i ) represents the i-th graphical code block, Indicates that the i-th graphical code block is closely related to the teaching knowledge, C represents the set of graphical code blocks, is the collection of all code blocks, Indicates educational objectives.

5. The graphical creative programming support method for classroom environment according to claim 1, characterized in that: The prediction module uses a normalized Levenshtein edit distance algorithm to predict between pseudocode and graphical code blocks.

6. The graphical creative programming support method for classroom environment according to claim 5, characterized in that: The expression of the Levenshtein edit distance algorithm is as follows: ID min =argmin true_block∈true blocks (gen_block,true_block) Visualization=get_block_image(ID min ) Among them, x, y represent the generated code block and the real code block, ID min The ID of the matched code block.

7. The graphical creative programming support method for classroom environment according to claim 1, characterized in that: The generation model also includes a prediction output module, which includes image output and voice output. The image output generates a corresponding graphical code block based on a Canny edge detector, and the voice output uses a stable diffusion algorithm to convert the text of the graphical code block into audio.

8. A graphical creative programming support system, characterized in that: The method is implemented by the method for supporting graphical creative programming in a classroom environment according to any one of claims 1 to 7, comprising: An input module, used for inputting data, including voice data, text data and image data; An interaction module, used to interact with the user based on input data to obtain interaction information; A database module for storing pre-built standard code collections; A data analysis module is used to analyze the interactive information to obtain the corresponding graphical code block from the database module; The result output module is used to output the obtained graphical code blocks in one or more forms of images, text or audio.

Images