Teaching information pushing method, teaching information generating method, teaching information model training method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a teaching information pushing method, a teaching information generating method, a model training method, electronic equipment and a storage medium. The teaching information pushing method comprises the following steps: receiving teaching data input by a teacher through an instant messaging platform of a client, wherein the teaching data comprises at least one of multi-modal teaching instant messaging information and multi-modal teaching task information; obtaining teacher attribute information and teaching scene information; generating prompt information according to the teaching data, the teacher attribute information and the teaching scene information; and based on the prompt information, teaching indication information responding to the teaching data is generated through a multi-modal generation model, and the teaching indication information is displayed through the instant messaging platform. According to the scheme, teaching indication information can be generated.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a teaching information pushing method, a generating method, a model training method, an electronic device and a storage medium. Background Art

[0002] In a typical home-school communication scenario, the school or teacher will establish a communication group chat (such as a class group) including students, parents and teachers on the instant messaging platform. Teachers can publish teaching instructions in the group chat on the instant messaging platform to communicate with students and parents according to teaching needs, so that students and parents can carry out corresponding learning or task execution. However, the generation effect of teachers' teaching instructions at this stage still needs to be improved. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a teaching information push method, generation method, model training method, electronic device and storage medium to at least partially solve the above problems.

[0004] According to the first aspect of an embodiment of the present application, a teaching information push method is provided, comprising: receiving teaching data input by a teacher through an instant messaging platform of a client, the teaching data comprising at least one of the following: multimodal teaching instant messaging information, multimodal teaching task information; obtaining teacher attribute information and teaching scene information; generating prompt information according to the teaching data, the teacher attribute information and the teaching scene information; based on the prompt information, generating teaching instruction information in response to the teaching data through a multimodal generation model, and displaying the teaching instruction information through the instant messaging platform.

[0005] According to a second aspect of an embodiment of the present application, a model training method is provided, comprising: obtaining training samples for model training, wherein the training samples include: teaching data samples, teacher attribute information samples and teaching scene information samples, and the teaching data samples include at least one of the following: multimodal teaching instant messaging information samples, multimodal teaching task information samples; generating prompt information samples according to the teaching data samples, the teacher attribute information samples and the teaching scene information samples; inputting the prompt information samples into a pre-trained multimodal generation model, and fine-tuning the multimodal generation model, wherein the multimodal generation model is used to generate teaching instruction information based on the teaching data.

[0006] According to the third aspect of an embodiment of the present application, a teaching information generation method is provided, comprising: receiving teaching data input through an instant messaging platform of a client, the teaching data comprising at least one of the following: multimodal teaching instant messaging information, multimodal teaching task information; sending the teaching data to a server to pass through the server, and processing the data based on prompt information through a multimodal generation model of the server to generate teaching instruction information, wherein the prompt information is generated according to the teaching data; and displaying the teaching instruction information through the instant messaging platform.

[0007] According to a fourth aspect of an embodiment of the present application, there is provided an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; the memory is used to store a computer program; the processor is used to execute any one of the methods described in the first aspect, the second aspect and the third aspect by running the computer program stored in the memory.

[0008] According to a fifth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in any one of the first aspect, the second aspect, and the third aspect is implemented.

[0009] According to a sixth aspect of an embodiment of the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements the method described in any one of the first aspect, the second aspect, and the third aspect.

[0010] According to the teaching information push scheme provided by the embodiment of the present application, it is possible to receive the teaching data input by the teacher through the instant messaging platform of the client, and the teaching data includes at least one of the multimodal teaching instant messaging information and the multimodal teaching task information, and then obtain the teacher attribute information and the teaching scene information, and then use the teaching data, the teacher attribute information and the teaching scene information to generate the prompt information, and then based on the prompt information, the teaching instruction information in response to the teaching data can be generated through the multimodal generation model, and the teaching instruction information can be displayed through the instant messaging platform. Thus, on the one hand, this scheme can realize the processing of the multimodal teaching data input by the teacher through the instant messaging platform of the client, and effectively generate the teaching instruction information through the multimodal generation model, and effectively improve the generation effect of the teaching instruction information; on the other hand, this scheme is to use the prompt information generated by the teaching data, the teacher attribute information and the teaching scene information to input the multimodal generation model for processing, and then use the powerful generation ability of the multimodal generation model to generate the teaching instruction information, which can better realize the content recommendation and generation assistance with high relevance to the teaching scene of home-school communication, and can effectively improve the accuracy and output efficiency of the teaching instruction information. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0012] Figure 1 A schematic diagram of an exemplary system applicable to the embodiment scheme of the present application.

[0013] Figure 2 The present invention is a flowchart of a method for pushing teaching information according to an embodiment of the present application.

[0014] Figure 3 A schematic diagram of the optional process of obtaining teacher attribute information and teaching scenario information in an embodiment of the present application.

[0015] Figure 4 The present invention is a flowchart of a method for generating teaching information according to an embodiment of the present application.

[0016] Figure 5 The present invention is a flowchart of a model training method according to an embodiment of the present application.

[0017] Figure 6 A schematic diagram of an optional process for obtaining training samples according to an embodiment of the present application.

[0018] Figure 7 A schematic diagram of an example scenario of an embodiment of the present application.

[0019] Figure 8 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0021] Figure 1 An exemplary system applicable to the embodiment of the present application is shown. Figure 1 As shown, the system 100 may include a cloud service end 102, a communication network 104 and / or one or more user devices 106. Figure 1 The example in FIG. 1 is a plurality of user devices.

[0022] The cloud service end 102 may be any suitable device for storing information, data, programs and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud service end clusters, etc. In some embodiments, the cloud service end 102 may perform any suitable function. For example, in some embodiments, the cloud service end 102 may generate teaching instruction information in response to teaching data. For example, the cloud service end 102 may receive teaching data input by a teacher through an instant messaging platform of a client, and the teaching data may include at least one of the following: multimodal teaching instant messaging information, multimodal teaching task information; obtaining teacher attribute information and teaching scene information; and then generating prompt information based on the teaching data, teacher attribute information and teaching scene information; and then generating teaching instruction information in response to the teaching data through a multimodal generation model based on the prompt information, and displaying the teaching instruction information through an instant messaging platform. As an optional example, in some embodiments, a multimodal generation model is deployed in the cloud service end 102, and the multimodal generation model may be obtained based on a general multimodal generation basic model. In some embodiments, the cloud service end 102 may also fine-tune the pre-trained multimodal generation model to obtain a multimodal generation model that can be used to generate teaching instruction information based on teaching data. As an optional example, in some embodiments, when performing model fine-tuning training, the cloud service end 102 may first obtain training samples for model training, wherein the training samples include: teaching data samples, teacher attribute information samples, and teaching scene information samples, and the teaching data samples include at least one of the following: multimodal teaching instant messaging information samples, multimodal teaching task information samples; and then, according to the teaching data samples, the teacher attribute information samples, and the teaching scene information samples, a prompt information sample is generated; thereafter, the prompt information sample is input into the pre-trained multimodal generation model, and the multimodal generation model is fine-tuned, wherein the multimodal generation model is used to generate teaching instruction information based on teaching data. Optionally, after generating the teaching instruction information, the cloud service end 102 may publish the generated teaching instruction information through the instant messaging platform of the client. The client may be the user device 106. The cloud service end 102 may be a server.

[0023] In some embodiments, the communication network 104 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 104 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN) and / or any other suitable communication network. The user device 106 can be connected to the communication network 104 via one or more communication links (e.g., communication link 112), and the communication network 104 can be linked to the cloud service end 102 via one or more communication links (e.g., communication link 114). The communication link can be any communication link suitable for transmitting data between the user device 106 and the cloud service end 102, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link, or any suitable combination of such links.

[0024] The user device 106 may include any one or more user devices suitable for presenting information, interacting with a user, etc. The user device 106 may be a client, provided with an instant messaging platform. As an optional example, in some embodiments, the user device 106 may first receive teaching data input through the instant messaging platform, and the teaching data includes at least one of the following: multimodal teaching instant messaging information, multimodal teaching task information; and then send the teaching data to the server (such as the cloud server 102) to pass through the server, and through the multimodal generation model of the server to process based on the prompt information, generate teaching instruction information, wherein the prompt information is generated according to the teaching data; and then display the teaching instruction information through the instant messaging platform. In some embodiments, the user device 106 may include any suitable type of device. For example, in some embodiments, the user device 106 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, and / or any other suitable type of user device.

[0025] Based on the above system, the embodiments of the present application provide a teaching information push scheme, a teaching information generation method and a model training scheme, which are described below through multiple embodiments.

[0026] Figure 2 1 is a flowchart of a method for pushing teaching information according to an embodiment of the present application. Optionally, the method can be used on the server side. Figure 2As shown, the teaching information push method may include the following steps:

[0027] S202: receiving teaching data input by the teacher through the instant messaging platform of the client.

[0028] In the embodiment of the present application, the teaching data includes at least one of the following: multimodal teaching instant messaging information and multimodal teaching task information.

[0029] In some optional application modes, the instant messaging platform can be used for communication and exchange between teachers, students, parents, etc. For example, a group chat (such as a class group) can be set up in the instant messaging platform, and teachers, students, parents, etc. can communicate and exchange about learning, life, etc. through group chat. For example, teachers can send instant messaging information, teaching task information, etc. to students, parents, etc. through group chat, and can also publish teaching instruction information through group chat to instruct students, parents, etc. to study accordingly according to the teaching instruction information.

[0030] It is understandable that the teaching instant messaging information or teaching task information sent by the teacher group through the group chat on the instant messaging platform can form a large amount of vertical scene corpus sedimentation, which can facilitate the generation of subsequent teaching instruction information and can also be used for the training and fine-tuning of multimodal generative models.

[0031] The teaching data sent through the instant messaging platform can be multimodal and can include multiple different types of data. By analyzing and processing the multimodal teaching data input by the teacher through the instant messaging platform, it is convenient to obtain teaching instruction information more accurately later.

[0032] Teaching instant messaging information may include messages input by teachers in the interface through various input methods of the instant messaging platform. For example, the input methods may include: input bar input, photo input, forwarding input, voice input, upload input, etc.

[0033] In some optional embodiments, the multimodal teaching instant messaging information may include at least one of the following: text, pictures, and files related to teaching. It should be understood that such multimodal teaching instant messaging information facilitates the subsequent accurate generation of prompt information that meets the needs of teachers, facilitates the subsequent more accurate acquisition of teaching instruction information, and improves the generation effect of teaching instruction information.

[0034] For example, text related to teaching can be input by methods including but not limited to entering text into an input bar, forwarding input text, and voice input text; pictures related to teaching can be input by methods including but not limited to entering pictures into an input bar, taking photos to input pictures, and forwarding input pictures; files related to teaching can be input by methods including but not limited to forwarding input files and uploading input files.

[0035] Teaching task information may include relevant information of teaching tasks sent by teachers through instant messaging platforms. For example, some example teaching tasks may include: homework tasks, check-in tasks, safety learning tasks, etc. The sending of teaching tasks may be achieved in any way, such as sending through a preset teaching task setting function (which may be sent directly by the teacher or automatically sent after the teacher has set it), publishing by entering in the input bar, publishing by forwarding, etc.

[0036] In some optional embodiments, the multimodal teaching task information may include: task links related to teaching.

[0037] The above-mentioned teaching-related task links may be links that can jump to a predetermined teaching task page to display content after being triggered. For example, in some examples, they may include but are not limited to links that record the task content of homework tasks, links that record the task content of punch-in tasks, links that record the task content of safety learning tasks, etc.

[0038] It should be understood that multimodal teaching task information including task links facilitates the subsequent accurate generation of prompt information that meets the teacher's needs, facilitates more accurate acquisition of teaching instruction information, and improves the generation effect of teaching instruction information.

[0039] For example, a teacher can use the preset teaching task setting function and set it as needed to generate a task link related to teaching, and send the task link to the instant messaging platform. After that, students or parents can click on the task link to jump to another teaching task page to view the content of the teaching task.

[0040] S204: Obtain teacher attribute information and teaching scenario information.

[0041] The teacher's attribute information and teaching scene information can be injected for subsequent generation of prompt information, so that the subsequent multimodal generation model can effectively generate teaching instruction information.

[0042] Optionally, after obtaining the teaching data, teacher attribute information and teaching scenario information may be acquired based on the teaching data.

[0043] In the embodiment of the present application, the teacher attribute information may be the teacher's attribute information related to teaching. For example, in some optional embodiments, the teacher attribute information may include at least one of: information about the subject taught by the teacher, information about the region where the teacher is located, and knowledge information about the teaching materials in the region where the teacher is located.

[0044] For example, information about subjects taught by teachers may include, but is not limited to, information about Chinese, mathematics, English, physics, chemistry, biology, history, geography, etc. This information can be used to adapt to the differences in subjects taught by different teachers, thereby facilitating the improvement of the effect of subsequently generated teaching instruction information.

[0045] For example, the information of the teacher's location may include but is not limited to at least one of the country, province, city, district, county, etc. This information can be used to adapt to the differences in the locations of different teachers, thereby improving the effect of the subsequently generated teaching instruction information.

[0046] For example, the knowledge information of the textbooks in the region where the teacher is located can indicate the knowledge context of the textbooks in the region where the teacher is located. Using the knowledge information of the textbooks in the region where the teacher is located can adapt to the differences in the knowledge content of the textbooks used by different teachers, thereby improving the effect of the subsequently generated teaching instruction information.

[0047] It should be understood that all the above information can effectively characterize the teacher. By obtaining such teacher attribute information, it is convenient to accurately generate prompt information that meets the teacher's needs, which is conducive to more accurate acquisition of teaching instruction information and improving the generation effect of teaching instruction information.

[0048] The teaching scene information may be information related to the external natural scene and / or social scene of the teacher's teaching. In some optional embodiments, the teaching scene information may include: at least one of external environment information and external event information.

[0049] The external environment information may be information related to the external natural environment of the area where the teacher is located, for example, it may include but is not limited to at least one of the relevant information such as weather, temperature, humidity, ultraviolet intensity, air pressure, visibility, natural disasters, etc. The use of this information can adapt to the differences in external natural environment factors in different areas where different teachers are located, thereby improving the effect of the subsequently generated teaching instruction information.

[0050] External event information may include, but is not limited to: external accidents (such as students drowning, students suffering from heat stroke, students having car accidents, etc.), social events (such as sudden epidemics, sudden malicious wounding incidents, organization of public welfare activities, etc.), teaching regulations change events (such as the promulgation / modification of systems / policies / regulations of education departments at all levels, modification of teaching materials, etc.), etc. The use of this information can adapt to the differences in different external event factors, thereby improving the effect of the subsequent generated teaching instruction information.

[0051] It should be understood that the above-mentioned various information can effectively indicate the required teaching scenarios. By obtaining such teaching scenario information, it is possible to accurately generate prompt information that meets the needs of teachers, which is conducive to more accurate acquisition of teaching instruction information and improving the generation effect of teaching instruction information.

[0052] The implementation method of step S204 is not specifically limited in the present application. Figure 3 As shown in the flowchart, an exemplary process of obtaining teacher attribute information and teaching scene information, i.e., step S204, may include the following steps:

[0053] S2042: Determine the teacher's identification information.

[0054] The teacher's identification information may be any appropriate information that can uniquely identify the teacher, such as a teacher number, a unique identification ID, etc.

[0055] In an optional embodiment, the teacher's work identity information may be obtained, and the teacher's identification information may be determined based on the work identity information. The teacher's work identity information may be information used to indicate the teacher's work identity. Optionally, the work identity information may be pre-stored in any storage space and may be directly obtained when needed. For example, the storage space may include but is not limited to a memory, a database, etc.

[0056] In some other optional embodiments, the identification information of the teacher can be determined based on the teaching data. For example, the teaching data input by the teacher can carry the identification information of the teacher, and the identification information of the teacher can be determined by analyzing the teaching data.

[0057] S2044: Based on the identification information, the teacher attribute information of the teacher is obtained through the application program interface, and the teaching scene information is obtained through the retrieval enhanced generation method.

[0058] Optionally, the teacher attribute information of the teacher can be stored in a certain storage space, for example, the teacher attribute information can be stored in a memory or a database, or can also be stored in a server. According to the identification information, a preset application programming interface (Application Programming Interface, API) can be called to obtain the returned teacher attribute information. For example, the teacher attribute information can include at least one of the information of the subject taught by the teacher, the information of the region where the teacher is located, and the knowledge information of the teaching materials in the region where the teacher is located.

[0059] Optionally, a knowledge base of teaching scene information can be pre-set, and various teaching scene information can be stored in the knowledge base. The knowledge base can obtain various teaching scene information from any channel, including but not limited to input by users, obtained from the Internet, obtained from instant messaging messages on instant messaging platforms, and so on. Various types of data including teaching scene information can be stored in the knowledge base in the form of vectors. For example, after obtaining teaching scene information from various channels, the obtained information can be classified and preprocessed, and the text and image features therein can be feature extracted, and the extracted text feature information and image feature information can be vectorized to obtain text vectors and image vectors, and the text vectors and image vectors are stored in the knowledge base to achieve the required corpus preparation for subsequent use.

[0060] Optionally, according to the identification information of the teacher, retrieval-augmented generation (RAG) can be performed based on the knowledge base to obtain the corresponding teaching scene information. For example, the teaching scene information may include at least one of the external environment information and the external event information. For example, the vector data of the teaching scene information can be retrieved from the knowledge base through the RAG method, and then the vector data can be converted into text for subsequent generation of prompt information.

[0061] It should be understood that in the embodiment of the present application, by determining the teacher's identification information, and then obtaining the teacher's attribute information through the API based on the identification information, and obtaining the teaching scene information through the RAG method, it is possible to effectively obtain rich and accurate teacher attribute information and teaching scene information, which is convenient for the subsequent accurate generation of prompt information that meets the teacher's needs, and is conducive to more accurate acquisition of teaching instruction information in the subsequent period, thereby improving the generation effect of teaching instruction information.

[0062] It should be noted that the present application does not limit the execution order of the above step S202 of acquiring teaching data and the teacher attribute information and teaching scene information in step S204. In some embodiments, the teaching data may be acquired first, and then the teacher attribute information and teaching scene information may be acquired (the order of acquiring the two is not specifically limited in the present application); in other embodiments, the teacher attribute information and teaching scene information may be acquired first (the order of acquiring the two is not specifically limited in the present application), and then the teaching information may be acquired. In still other embodiments, any number of the teaching data, teaching attribute information, and teaching scene information may be acquired in parallel.

[0063] S206: Generate prompt information based on teaching data, teacher attribute information and teaching scenario information.

[0064] Optionally, a prompt message prompt can be generated based on teaching data, teacher attribute information, and teaching scenario information to facilitate subsequent input into a multimodal generation model. Optionally, when generating a prompt message prompt, it can be generated according to a preset prompt message format. For example, an information header and / or information footer indicating the information, or other information related to actual needs can be added to each type of information.

[0065] In some optional embodiments, step S206 may include: if the teaching data includes at least one of text and pictures in instant messaging, generating prompt information based on at least one of the text and pictures, teacher attribute information and teaching scene information.

[0066] For example, a prompt message in the following format can be generated (the following "xxx" can represent text or image):

[0067]

[0068] In some optional embodiments, step S206 may include: if the teaching data includes a file in an instant message, obtaining the file content, and generating prompt information according to the file content, teacher attribute information and teaching scenario information.

[0069] It should be understood that by extracting the file content from the file and then using the file content to conveniently generate prompt information, so as to improve the generation effect of the teaching instruction information.

[0070] The text content in the file can be extracted in any manner, such as by algorithm extraction, large model analysis, etc. In some optional embodiments, the file content can be parsed to obtain key information, and prompt information can be generated based on the key information, teacher attribute information, and teaching scenario information.

[0071] The key information may be information related to the subject in the file content. In one embodiment, the key information may be summary information of the file content. The file content may be parsed in any manner, for example, the file content may be summarized and summarized by algorithm processing, large model parsing, etc., to obtain the key information of the file content.

[0072] Optionally, after the key information of the file content is obtained, the prompt information is generated by combining the key information of the file content, the teacher attribute information and the teaching scenario information to facilitate the subsequent input of the multimodal generation model.

[0073] It should be understood that in the embodiment of the present application, the key information is obtained by parsing the file content. When the prompt information is generated by the key information, on the one hand, the accuracy of the generated prompt information can be guaranteed. On the other hand, since the key information is generally more concise, compared with the prompt information generated directly from the file content, the amount of calculation when the prompt information is subsequently input into the multimodal generation model for processing to generate the teaching instruction information can be better reduced. Therefore, by generating the prompt information in this way, it is easy to improve the generation effect of the teaching instruction information.

[0074] In some optional embodiments, step S206 may include: if the teaching data includes a task link, obtaining task information according to the task link, and generating prompt information according to the task information, teacher attribute information and teaching scene information.

[0075] As mentioned above, the above-mentioned task link can be a link that can jump to a predetermined teaching task page to display content after being triggered. For example, in some examples, it may include but is not limited to a link that records the task content of homework tasks, a link that records the task content of punch-in tasks, and a link that records the task content of safety learning tasks. Task information can be determined based on the content related to the teaching task recorded in the task link. Task information can be obtained from the task link in any way, for example, the task link can be analyzed by algorithm processing, large model analysis, etc. to obtain the task information therein.

[0076] Optionally, after the task information of the task link is obtained, prompt information is generated by combining the task information, teacher attribute information and teaching scenario information to facilitate subsequent input into the multimodal generation model.

[0077] It is understandable that the optional methods of the above step S206 can be based on the content of the teaching data (such as pictures, texts, files, task links, etc.), and one or more of them can be selected as needed.

[0078] S208: Based on the prompt information, generate teaching instruction information in response to the teaching data through a multimodal generation model, and display the teaching instruction information through an instant messaging platform.

[0079] The teaching instruction information is used to instruct students and parents to perform corresponding learning or task execution. Some optional types of teaching instruction information may be similar to homework, activity initiatives, safety education instructions, etc.

[0080] Optionally, the teaching instruction information generated by the multimodal generation model may also be multimodal.

[0081] After the prompt information is obtained, the prompt information can be input into the multimodal generation model, and the multimodal generation model performs generation processing according to the prompt information, thereby generating teaching instruction information responsive to the teaching data. The server can display the teaching instruction information generated by the multimodal generation model through the instant messaging platform, so that the teaching instruction information can be displayed to at least one of the teacher, the student, and the parent through the instant messaging platform (such as a group chat therein), so as to better conduct teaching through the instant messaging platform.

[0082] The multimodal generation model in the embodiment of the present application can be obtained based on a general multimodal generation basic model. In some embodiments, the server can also perform fine-tuning training on the pre-trained multimodal generation model to obtain a multimodal generation model that can be used to generate teaching instruction information based on teaching data. An optional fine-tuning training scheme is introduced in the model training method embodiment below and will not be repeated here.

[0083] Based on this, the optional teaching information push scheme provided by the embodiment of the present application can receive the teaching data input by the teacher through the instant messaging platform of the client, and the teaching data includes at least one of the multimodal teaching instant messaging information and the multimodal teaching task information, and then obtain the teacher attribute information and the teaching scene information, and then use the teaching data, the teacher attribute information and the teaching scene information to generate the prompt information, and then based on the prompt information, the teaching instruction information in response to the teaching data can be generated through the multimodal generation model, and the teaching instruction information can be displayed through the instant messaging platform. Thus, on the one hand, this scheme can realize the processing of the multimodal teaching data input by the teacher through the instant messaging platform of the client, and effectively generate the teaching instruction information through the multimodal generation model, and effectively improve the generation effect of the teaching instruction information; on the other hand, this scheme is to use the prompt information generated by the teaching data, the teacher attribute information and the teaching scene information to input the multimodal generation model for processing, and then use the powerful generation ability of the multimodal generation model to generate the teaching instruction information, which can better realize the content recommendation and generation assistance with high relevance to the teaching scene of home-school communication, and can effectively improve the accuracy and output efficiency of the teaching instruction information.

[0084] Figure 4 1 is a flowchart of a method for generating teaching information according to an embodiment of the present application. Optionally, the method can be used on a client. Figure 4 As shown, the teaching information generation method may include the following steps:

[0085] S402: receiving teaching data inputted via the instant messaging platform of the client.

[0086] In the embodiment of the present application, the teaching data includes at least one of the following: multimodal teaching instant messaging information and multimodal teaching task information.

[0087] S404: Send the teaching data to the server, so as to pass through the server, and process the prompt information through the multimodal generation model of the server to generate teaching instruction information.

[0088] In the embodiment of the present application, the prompt information is generated based on the teaching data.

[0089] S406: Display teaching instruction information via the instant messaging platform.

[0090] Based on this, on the one hand, this solution can realize the processing of multimodal teaching data input by teachers through the client's instant messaging platform, and effectively generate teaching instruction information through a multimodal generation model, thereby effectively improving the generation effect of teaching instruction information; on the other hand, this solution uses prompt information generated by teaching data to input into the multimodal generation model for processing, and then uses the powerful generation capability of the multimodal generation model to generate teaching instruction information, which can better realize content recommendation and generation assistance that is highly relevant to the home-school communication teaching scenario, and can effectively improve the accuracy and output efficiency of teaching instruction information.

[0091] In some optional embodiments, prompt information is generated based on teaching data, teacher attribute information and teaching scene information.

[0092] In some optional embodiments, teacher attribute information is obtained through an application program interface based on the teacher's identification information, and teaching scene information is obtained through a retrieval enhanced generation method based on the teacher's identification information.

[0093] In some optional embodiments, the multimodal teaching instant messaging information includes at least one of the following: text, pictures, files related to teaching; and / or the multimodal teaching task information includes: task links related to teaching.

[0094] In some optional embodiments, if the teaching data includes a file in an instant message, the prompt information is generated based on the file content, teacher attribute information and teaching scenario information of the file.

[0095] In some optional embodiments, the prompt information is generated based on key information, teacher attribute information and teaching scenario information, wherein the key information is obtained by parsing the file content of the file.

[0096] In some optional embodiments, if the teaching data includes a task link, the prompt information is generated according to the task information, the teacher attribute information and the teaching scene information, wherein the task information is obtained according to the task link.

[0097] In some optional embodiments, the teacher attribute information includes at least one of information about the subjects taught by the teacher, information about the region where the teacher is located, and knowledge information about the teaching materials in the region where the teacher is located; and / or, the teaching scene information includes at least one of external environment information and external event information.

[0098] It should be understood that the relevant content and beneficial effects of the teaching information generating method in the embodiment of the present application have been described in detail in the teaching information pushing method embodiment in the previous text, which can be understood by referring to the previous text and will not be repeated here.

[0099] Figure 5 1 is a flowchart of the steps of a model training method according to an embodiment of the present application. Figure 5 As shown, the model training method may include the following steps:

[0100] S502: Obtain training samples for model training.

[0101] In an embodiment of the present application, the training samples include: teaching data samples, teacher attribute information samples and teaching scene information samples; the teaching data samples include at least one of the following: multimodal teaching instant messaging information samples, multimodal teaching task information samples.

[0102] In this embodiment, the training samples can be obtained in any appropriate manner, for example, from a sample database, etc. Among them, for the teaching data samples, they can be teaching instant messaging information and / or teaching task information obtained in any appropriate manner. On this basis, teacher attribute information samples and teaching scenario information samples are obtained.

[0103] Optionally, teacher attribute information samples and teaching scene information samples can be obtained based on the teaching data samples.

[0104] Instructional data samples can be multimodal and can include many different types of data.

[0105] In the embodiment of the present application, the teacher attribute information sample may be a sample of attribute information related to the teacher's teaching. In some optional embodiments, the teacher attribute information sample may include at least one of the information sample of the subject taught by the teacher, the information sample of the region where the teacher is located, and the knowledge information sample of the teaching materials in the region where the teacher is located.

[0106] For example, information samples of subjects taught by teachers may include, but are not limited to, samples of relevant information such as Chinese, mathematics, English, physics, chemistry, biology, history, geography, etc.

[0107] For example, the information sample of the teacher's region may include, but is not limited to, at least one of samples of information such as the country, province, city, district, and county.

[0108] For example, the knowledge information sample of the teaching materials in the teacher's area may be a sample that can indicate the knowledge context of the teaching materials in the teacher's area.

[0109] It should be understood that the use of the above information samples can effectively characterize teachers. By obtaining such teacher attribute information samples, it is convenient to subsequently accurately generate prompt information samples that meet the needs of teachers, which is conducive to fine-tuning the multimodal generation model, so that the multimodal generation model after fine-tuning training can more accurately generate teaching instruction information and improve the generation effect of teaching instruction information.

[0110] The teaching scene information sample may be a sample of information related to the external natural scene and / or social scene of the teacher's teaching. In some optional embodiments, the teaching scene information sample may include at least one of an external environment information sample and an external event information sample.

[0111] The external environment information sample may be a sample of information related to the external natural environment of the teacher's location, for example, including but not limited to at least one of samples of relevant information such as weather, temperature, humidity, ultraviolet intensity, air pressure, visibility, natural disasters, etc.

[0112] External event information samples may include, but are not limited to: external accidents (such as students drowning, students suffering from heatstroke, students getting into car accidents, etc.), social events (such as sudden outbreaks of epidemics, sudden serious injuries, organization of public welfare activities, etc.), teaching regulations change events (such as the promulgation / modification of systems / policies / regulations of education departments at all levels, modification of textbook content, etc.), and other related information samples.

[0113] It should be understood that the above-mentioned various information samples can effectively indicate the required teaching scenarios. By obtaining such teaching scenario information samples, it is possible to accurately generate prompt information samples that meet the needs of teachers in the subsequent process, which is conducive to fine-tuning the multimodal generation model. The multimodal generation model after fine-tuning training can generate teaching instruction information more accurately and improve the generation effect of teaching instruction information.

[0114] In some optional embodiments, the multimodal teaching instant messaging information samples may include at least one of the following: text samples, picture samples, and file samples related to teaching. It should be understood that such multimodal teaching instant messaging information samples facilitate the subsequent accurate generation of prompt information samples that meet the needs of teachers, and are conducive to fine-tuning the multimodal generation model, so that the multimodal generation model after fine-tuning training can more accurately generate teaching instruction information and improve the generation effect of teaching instruction information.

[0115] In some optional embodiments, the multimodal teaching task information samples may include: teaching-related task link samples.

[0116] The above-mentioned teaching-related task link samples may be samples of links that can jump to a predetermined teaching task page to display content after being triggered. For example, in some examples, they may include but are not limited to links that record the task content of homework tasks, links that record the task content of punch-in tasks, links that record the task content of safety learning tasks, etc.

[0117] It should be understood that by including multimodal teaching task information samples with task links, it is convenient to subsequently accurately generate prompt information samples that meet the needs of teachers, which is conducive to fine-tuning the multimodal generation model, so that the multimodal generation model after fine-tuning training can more accurately generate teaching instruction information and improve the generation effect of teaching instruction information.

[0118] In some optional embodiments, referring to Figure 6 As shown in the flowchart, the process of obtaining the above training samples, i.e., the process of step S502, may include the following steps:

[0119] S5022: Determine the teacher's identification information sample.

[0120] The teacher's attribute information samples and teaching scene information samples can be injected for the subsequent generation of prompt information samples, which facilitates the subsequent fine-tuning training of the multimodal generation model, so that the multimodal generation model after fine-tuning training can effectively generate teaching instruction information.

[0121] Optionally, after obtaining the teaching data sample, a teacher attribute information sample and a teaching scene information sample may be acquired based on the teaching data sample.

[0122] The teacher's identification information sample may be any appropriate sample of information that can uniquely identify the teacher, such as a teacher number, a unique identification ID, and the like.

[0123] In an optional embodiment, a teacher's work identity information sample may be obtained, and the teacher's identification information sample may be determined based on the work identity information sample. The teacher's work identity information sample may be a sample of information indicating the teacher's work identity. Optionally, the work identity information sample may be pre-stored in any storage space and may be directly obtained when needed. For example, the storage space may include but is not limited to a memory, a database, etc.

[0124] In some other optional embodiments, the teacher's identification information sample can be determined based on the teaching data sample. For example, the teaching data sample input by the teacher can carry the teacher's identification information sample, and the teacher's identification information sample can be determined by analyzing the teaching data sample.

[0125] S5024: Based on the identification information sample, obtain the teacher attribute information sample of the teacher through the application program interface, and obtain the teaching scene information sample through the retrieval enhanced generation method.

[0126] Optionally, the teacher attribute information of the teacher can be stored in a certain storage space, for example, the teacher attribute information can be stored in a memory or a database, or can also be stored in a server. According to the identification information sample, a preset application programming interface (Application Programming Interface, API) can be called to obtain a returned teacher attribute information sample, which is used as a teacher attribute information sample. For example, the teacher attribute information sample can include at least one of the information sample of the subject taught by the teacher, the information sample of the region where the teacher is located, and the knowledge information sample of the teaching materials in the region where the teacher is located.

[0127] Optionally, a knowledge base of teaching scene information can be pre-set, and various teaching scene information can be stored in the knowledge base. The knowledge base and teaching scene information have been explained in the previous text, and can be understood with reference to the previous text. According to the teacher's identification information sample, retrieval-augmented generation (RAG) can be performed based on the knowledge base to obtain the corresponding teaching scene information, which is used as a teaching scene information sample. For example, the teaching scene information sample may include at least one of an external environment information sample and an external event information sample. For example, the vector data of the teaching scene information can be retrieved from the knowledge base by RAG, and then the vector data can be converted into text for the subsequent generation of prompt information samples.

[0128] It should be understood that in the embodiment of the present application, by determining the teacher's identification information sample, and then obtaining the teacher's attribute information sample through the API based on the identification information sample, and obtaining the teaching scene information sample through the RAG method, it is possible to effectively obtain rich and accurate teacher attribute information samples and teaching scene information samples, which is convenient for the subsequent accurate generation of prompt information samples that meet the teacher's needs, and is conducive to fine-tuning the multimodal generation model, so that the multimodal generation model after fine-tuning training can more accurately generate teaching instruction information and improve the generation effect of teaching instruction information.

[0129] S504: Generate prompt information samples according to the teaching data samples, teacher attribute information samples and teaching scene information samples.

[0130] Optionally, prompt information samples (prompt samples) can be generated based on teaching data samples, teacher attribute information samples and teaching scenario information samples, so as to facilitate subsequent input into the multimodal generation model for fine-tuning training. Optionally, when generating prompt information samples, they can be generated according to a preset prompt information format. For example, for each type of information, an information header and / or information footer indicating the information, or other information related to actual needs can be added. (An example format can be understood in conjunction with the example format of prompt information in the previous text)

[0131] In some optional embodiments, step S504 may include: if the teaching data sample includes at least one of a text sample and a picture sample in instant messaging, then generating a prompt information sample based on at least one of the text sample and the picture sample, a teacher attribute information sample and a teaching scene information sample.

[0132] In some optional embodiments, step S504 may include: if the teaching data sample includes a file sample in instant messaging, obtaining a file content sample, and generating a prompt information sample according to the file content sample, the teacher attribute information sample and the teaching scene information sample.

[0133] It should be understood that by extracting file content samples from file samples and then using the file content samples, prompt information samples can be easily generated to facilitate fine-tuning training of the multimodal generation model and improve the generation effect of the multimodal generation model on teaching instruction information.

[0134] The text content sample in the file sample can be extracted in any manner, such as by algorithm extraction, large model analysis, etc. In some optional embodiments, the file content sample can be analyzed to obtain a key information sample, and a prompt information sample can be generated based on the key information sample, the teacher attribute information sample, and the teaching scenario information sample.

[0135] The key information sample may be a sample of information related to the subject in the file content sample. In one embodiment, the key information sample may be summary information of the file content sample. The file content sample may be parsed in any manner, for example, the file content sample may be summarized and summarized by algorithm processing, large model parsing, etc., to obtain the key information sample of the file content sample.

[0136] Optionally, after obtaining the key information of the file content, the key information of the file content, the teacher attribute information sample and the teaching scene information sample are combined to generate a prompt information sample (prompt sample) to facilitate subsequent input into the multimodal generation model for fine-tuning training.

[0137] It should be understood that in the embodiment of the present application, the key information sample is obtained by parsing the file content sample. When the prompt information is generated by the key information sample, on the one hand, the accuracy of the generated prompt information sample can be guaranteed. On the other hand, because the key information sample is generally more concise, compared with the prompt information sample generated directly by the file content sample, the amount of calculation when the prompt information sample is subsequently input into the multimodal generation model for fine-tuning training can be better reduced, thereby helping to improve the effect of fine-tuning training. Therefore, by generating prompt information samples in this way, it is convenient to fine-tune the multimodal generation model and improve the generation effect of the multimodal generation model on teaching instruction information.

[0138] In some optional embodiments, step S504 may include: if the teaching data sample includes a task link sample, obtaining a task information sample according to the task link sample, and generating prompt information according to the task information sample, the teacher attribute information sample and the teaching scene information sample.

[0139] As mentioned above, the above-mentioned task link samples may be samples of links that can jump to a predetermined teaching task page to display content after being triggered. For example, in some examples, it may include but is not limited to links that record the task content of homework tasks, links that record the task content of punch-in tasks, and links that record the task content of safety learning tasks. The task information sample can be determined based on the content of the information related to the teaching task recorded in the task link sample. The task information sample can be obtained from the task link sample in any manner, for example, the task link sample can be analyzed by algorithm processing, large model analysis, etc. to obtain the task information sample therein.

[0140] Optionally, after the task information sample of the task link sample is obtained, the task information sample, the teacher attribute information sample and the teaching scene information sample are combined to generate a prompt information sample (prompt sample) to facilitate subsequent input into the multimodal generation model.

[0141] It is understandable that the optional methods of the above step S504 can be based on the content of the teaching data sample (such as picture samples, text samples, file samples, task link samples, etc.), and one or more of them can be selected as needed.

[0142] S506: Input the prompt information sample into the pre-trained multimodal generation model to perform fine-tuning training on the multimodal generation model.

[0143] In an embodiment of the present application, a multimodal generation model is used to generate teaching instruction information based on teaching data.

[0144] The pre-trained multimodal generation model already has the basic function of generating multimodal information based on multimodal data. Fine-tuning it on this basis can make the fine-tuned multimodal generation model better suitable for the teaching scenario of the embodiment of the present application. Thus, the prompt information sample can be input into the pre-trained multimodal generation model to perform multimodal fine-tuning training on the multimodal generation model.

[0145] Fine-tuning training can be performed for multiple rounds, and the end condition can be that the number of fine-tuning training rounds reaches a predetermined number of rounds, or the end condition can also be that the loss value of the model loss function reaches a certain preset value, then it is considered that the end condition of model fine-tuning is met. After the fine-tuning training is completed, a multimodal generation model that can more accurately generate teaching instruction information based on teaching data can be obtained.

[0146] Based on this, the optional model training scheme provided in the embodiment of the present application can obtain training samples for model training, and the training samples include: teaching data samples, teacher attribute information samples and teaching scene information samples, the teaching data samples include multimodal teaching instant messaging information samples, multimodal teaching task information samples, at least one of them, and then generate prompt information samples according to the teaching data samples, teacher attribute information samples and teaching scene information samples, and then the prompt information samples can be input into the pre-trained multimodal generation model, and the multimodal generation model is fine-tuned and trained, and the multimodal generation model is used to generate teaching instruction information based on teaching data. Thus, it is possible to effectively fine-tune the multimodal generation model, so that the multimodal generation model after fine-tuning training can accurately generate teaching instruction information based on teaching data, and improve the generation effect of teaching instruction information. In addition, in the subsequent use of the powerful generation ability of the multimodal generation model completed by fine-tuning training to generate teaching instruction information, it is possible to better achieve content recommendation and generation assistance with high relevance to the home-school communication teaching scene, and effectively improve the accuracy and output efficiency of teaching instruction information.

[0147] Figure 7 A schematic diagram of a scenario example of an embodiment of the present application. Figure 7The teaching information push scheme and generation scheme in the embodiment of the present application are shown as a whole. It should be noted that it should be understood that Figure 7 The example scenarios shown are only used to facilitate understanding of the embodiments of the present application and are not intended to limit the embodiments of the present application.

[0148] like Figure 7 As shown, the teacher inputs multimodal teaching instant messaging information (which may include text, pictures, files, etc.) and multimodal teaching task information (which may include but is not limited to task links of homework tasks, check-in tasks, safety learning tasks, etc.) in the group chat (such as class group) through the client's instant messaging platform, and then obtains the multimodal teaching instant messaging messages and multimodal teaching task information as teaching data, and sends the teaching data to the server.

[0149] Afterwards, the server can pre-process the obtained information, identify the teacher's identification, and then inject the teacher's attribute information and teaching scene information. For example, the teacher's identification information can be determined first, and then based on the identification information, the teacher's attribute information can be obtained through the API (for example, the teacher's attribute information may include at least one of the information about the subjects taught by the teacher, the information about the region where the teacher is located, and the knowledge information of the textbooks in the region where the teacher is located), and the teaching scene information can be obtained through the RAG method (for example, the teaching scene information may include at least one of the external environment information and the external event information).

[0150] Afterwards, the server can generate prompt information based on the teaching data, teacher attribute information and teaching scenario information.

[0151] Afterwards, the server can input the prompt information into the multimodal generation model, and the multimodal generation model can process the prompt information to obtain teaching instruction information.

[0152] Afterwards, the server can push the teaching instruction information generated by the multimodal generation model to the client, and display the teaching instruction information through the instant messaging platform. Displaying the teaching instruction information assisted by the multimodal generation model through the instant messaging platform can achieve effective information feedback to at least one of the teachers, students, parents, etc., which is conducive to improving the teaching effect of the home-school communication scenario.

[0153] For example, in an example application scenario, the class teacher in XX District, XX City can be informed through the instant messaging platform of the client that most of the class teachers in this district sent anti-drowning notices to parents before the summer vacation. Whether to send a notice to inform the parents of this class, after the class teacher agrees, the corresponding teaching data entered in the instant messaging platform can be obtained, and the title and content of the anti-drowning notice (that is, the teaching instruction information) can be assisted in generating through the multimodal generation model, and displayed on the client of the class teacher. The class teacher can choose to directly apply the teaching instruction information or can also adapt and modify the teaching instruction information, and can send the directly applied teaching instruction information or the adapted teaching instruction information to other students, parents, etc. through the group chat of the instant messaging platform to realize the teaching instruction of the anti-drowning notice. It should be understood that this is only an example for easy understanding and is not any limitation to the embodiments of the present application.

[0154] It can be seen that through the technical solution in this example, on the one hand, it is possible to process the multimodal teaching data input by the teacher through the client's instant messaging platform, and effectively generate teaching instruction information through the multimodal generation model, thereby effectively improving the generation effect of the teaching instruction information; on the other hand, since the prompt information generated by teaching data, teacher attribute information and teaching scene information is input into the multimodal generation model for processing, and then the powerful generation ability of the multimodal generation model is used to generate the teaching instruction information, it can better realize the content recommendation and generation assistance that is highly relevant to the home-school communication teaching scene, and can effectively improve the accuracy and output efficiency of the teaching instruction information.

[0155] It should also be understood that the above Figure 7 The related descriptions are only used to explain the technical solutions of the embodiments of the present application for easy understanding, and they are not intended to limit the embodiments of the present application in any way.

[0156] An embodiment of the present application also provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; the memory is used to store a computer program; and the processor is used to execute any method described in the aforementioned multiple method embodiments by running the computer program stored in the memory.

[0157] Figure 8 1000 is a block diagram of an optional electronic device in an embodiment of the present application. The present application embodiment does not limit the specific implementation of the electronic device 1000. Figure 8The electronic device 1000 provided in the embodiment of the present application includes: a processor 1002, a communication interface 1004, a memory 1006, and a communication bus 1008. Among them:

[0158] The processor 1002 , the communication interface 1004 , and the memory 1006 communicate with each other via a communication bus 1008 .

[0159] The communication interface 1004 is used to communicate with other electronic devices or servers.

[0160] The processor 1002 is used to execute the computer program 1010, and specifically can execute the relevant steps of any method embodiment in the aforementioned multiple method embodiments.

[0161] Specifically, the computer program 1010 may include program codes, which include computer operation instructions.

[0162] The processor 1002 may be a CPU, or a GPU (Graphic Processing Unit), or an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0163] The memory 1006 is used to store the computer program 1010. The memory 1006 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0164] The computer program 1010 may be specifically configured to enable the processor 1002 to execute a method of any one of the foregoing method embodiments.

[0165] The specific implementation of each step in the computer program 1010 can refer to the corresponding description of the corresponding steps and units in any of the above-mentioned multiple method embodiments, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above-mentioned method embodiments, which will not be repeated here.

[0166] In addition, the embodiment of the present application further provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method of any of the aforementioned multiple method embodiments is implemented. The computer storage medium includes, but is not limited to: a compact disc read-only memory (CD-ROM), a random access memory (RAM), a floppy disk, a hard disk, or a magneto-optical disk, etc.

[0167] An embodiment of the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements a method as in any one of the above-mentioned multiple method embodiments.

[0168] In addition, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used to train the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0169] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0170] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA)). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., a random access memory (RAM), a read-only memory (ROM), a flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown here.

[0171] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for specific applications, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0172] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". It should be noted that the concepts of "first", "second", etc. mentioned in the embodiments of the present application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units. It should be noted that the modifications of "one" and "multiple" mentioned in the embodiments of the present application are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0173] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A teaching information push method, comprising: Receiving teaching data input by a teacher through an instant messaging platform of a client, wherein the teaching data includes at least one of the following: multimodal teaching instant messaging information and multimodal teaching task information; Obtain teacher attribute information and teaching scenario information; Generate prompt information according to the teaching data, the teacher attribute information and the teaching scene information; Based on the prompt information, teaching instruction information responsive to the teaching data is generated through a multimodal generation model, and the teaching instruction information is displayed through the instant messaging platform.

2. The method according to claim 1, wherein: The obtaining of teacher attribute information and teaching scenario information includes: Determining identification information of the teacher; Based on the identification information, the teacher attribute information of the teacher is obtained through an application program interface, and the teaching scene information is obtained through a retrieval enhanced generation method.

3. The method according to claim 1 or 2, wherein: The multimodal teaching instant messaging information includes at least one of the following: text, pictures, and files related to teaching; and / or, Multimodal teaching task information includes: task links related to teaching.

4. The method according to claim 3, wherein: The generating prompt information according to the teaching data, the teacher attribute information and the teaching scene information includes: If the teaching data includes files in instant messaging, the file content is obtained, and prompt information is generated according to the file content, the teacher attribute information and the teaching scene information.

5. The method according to claim 4, wherein: The generating prompt information according to the file content, the teacher attribute information and the teaching scenario information includes: The file content is parsed to obtain key information, and prompt information is generated according to the key information, the teacher attribute information and the teaching scene information.

6. The method according to claim 3, wherein: The generating prompt information according to the teaching data, the teacher attribute information and the teaching scene information includes: If the teaching data includes a task link, task information is obtained according to the task link, and prompt information is generated according to the task information, the teacher attribute information and the teaching scene information.

7. The method according to claim 1 or 2, wherein: The teacher attribute information includes at least one of information about the subject taught by the teacher, information about the region where the teacher is located, and knowledge information about the teaching materials in the region where the teacher is located; and / or, The teaching scene information includes at least one of external environment information and external event information.

8. A model training method, comprising: Acquire training samples for model training, wherein the training samples include: teaching data samples, teacher attribute information samples and teaching scene information samples, and the teaching data samples include at least one of the following: multimodal teaching instant messaging information samples and multimodal teaching task information samples; Generate a prompt information sample according to the teaching data sample, the teacher attribute information sample and the teaching scene information sample; The prompt information sample is input into a pre-trained multimodal generation model, and the multimodal generation model is fine-tuned, wherein the multimodal generation model is used to generate teaching instruction information based on teaching data.

9. The method according to claim 8, wherein: The obtaining of training samples for model training includes: Determining a sample of identification information of the teacher; Based on the identification information sample, the teacher attribute information sample of the teacher is obtained through an application program interface, and the teaching scene information sample is obtained through a retrieval enhanced generation method.

10. The method according to claim 8 or 9, wherein: The teacher attribute information sample includes at least one of the information sample of the subject taught by the teacher, the information sample of the region where the teacher is located, and the knowledge information sample of the teaching materials in the region where the teacher is located; and / or, The teaching scene information sample includes at least one of an external environment information sample and an external event information sample; and / or, The multimodal teaching instant messaging information sample includes at least one of the following: a text sample, a picture sample, and a file sample related to teaching; and / or, Multimodal teaching task information samples include: teaching-related task link samples.

11. A method for generating teaching information, comprising: Receiving teaching data inputted through an instant messaging platform of a client, wherein the teaching data includes at least one of the following: multimodal teaching instant messaging information and multimodal teaching task information; Sending the teaching data to the server to be processed by the server and by a multimodal generation model of the server based on the prompt information to generate teaching instruction information, wherein the prompt information is generated according to the teaching data; The teaching instruction information is displayed via the instant messaging platform.

12. An electronic device comprising: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 11 by running the computer program stored in the memory.

13. A computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 11.

14. A computer program product, comprising a computer program, which implements the method according to any one of claims 1 to 11 when executed by a processor.

Images