Classroom chapter identification method, apparatus, device, and storage medium

Abstract

The application provides a classroom chapter identification method, device and equipment and a storage medium. The method comprises the following steps: obtaining a classroom content text, wherein the classroom content text comprises a classroom audio text and / or a classroom courseware content text corresponding to the classroom audio text, and the classroom audio text is obtained by automatic speech recognition on classroom audio; performing similarity matching on the classroom content text and chapter association text corresponding to each chapter name in a chapter search library to determine N candidate chapter names, wherein the chapter search library is used for storing a plurality of chapter names, the chapter association text is a text used for introducing chapter knowledge and associated with the chapter name, and the chapter association text corresponding to the N candidate chapter names has the highest matching degree with the classroom content text; and performing similarity matching on the N candidate chapter names and the classroom content text to determine a classroom chapter name, wherein the classroom chapter name is the candidate chapter name having the highest matching degree with the classroom content text. The application can guarantee the accuracy of classroom chapter identification.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to methods, apparatus, devices and storage media for classroom chapter recognition. Background Technology

[0002] With the continuous development and application of technology, smart education systems are gradually becoming a hot topic in the education field. Besides promoting the modernization and informatization of education, smart education systems also bring more refined management and personalized education, meeting the diverse needs of students and improving the quality of education.

[0003] In smart education systems, classroom chapter recognition helps integrate relevant resources to create interdisciplinary learning content, and also helps the system recommend relevant learning resources based on students' learning history and chapter recognition results. How to perform classroom chapter recognition has become a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for identifying classroom chapters.

[0005] Firstly, a method for identifying classroom chapters is provided, including:

[0006] The classroom content text is obtained, which includes classroom audio text and / or classroom courseware content text corresponding to the classroom audio text. The classroom audio text is obtained by performing automatic speech recognition (ASR) on the classroom audio.

[0007] The similarity of the classroom content text with the chapter-related text corresponding to each chapter name in the chapter search library is performed to determine N candidate chapter names. The chapter search library is used to store multiple chapter names. The chapter-related text is the text associated with the chapter name and used to introduce the chapter knowledge. The chapter-related text corresponding to the N candidate chapter names has the highest matching degree with the classroom content text. N is a positive integer greater than or equal to 2.

[0008] The N candidate chapter names are matched with the classroom content text for similarity, and the classroom chapter name is determined. The classroom chapter name is the candidate chapter name with the highest matching degree with the classroom content text.

[0009] In this technical solution, after obtaining the classroom content text, the text is first matched with the associated text corresponding to each chapter name in the chapter search database to determine N candidate chapter names. Since the associated text is text used to introduce the chapter knowledge associated with the chapter name, the associated text corresponding to the N candidate chapter names has the highest matching degree with the classroom content text, which can achieve fine ranking of the chapter names in the chapter search database and determine the multiple chapter names most likely to correspond to the classroom content. Then, the N candidate chapter names are matched with the classroom content text to determine the classroom chapter name. The classroom chapter name is the candidate chapter name with the highest matching degree with the classroom content text, which can achieve recall and re-ranking of chapter names, thereby determining the final classroom chapter name corresponding to the classroom content. By first finely ranking the chapter names according to the associated text corresponding to the chapter name and then recalling and re-ranking the chapter names according to the chapter name to determine the classroom chapter name corresponding to the classroom content, the accuracy of classroom chapter identification can be guaranteed.

[0010] In conjunction with the first aspect, in one possible implementation, obtaining the classroom content text includes: obtaining the classroom audio text and the corresponding classroom courseware content text; calculating the correlation between the classroom audio text and the classroom courseware content text; if the correlation is less than or equal to a first preset threshold, determining the classroom audio text as the classroom content text; if the correlation is greater than or equal to a second preset threshold, determining the classroom courseware content text as the classroom content text, wherein the second preset threshold is greater than the first preset threshold; and if the correlation is greater than the first preset threshold and less than the second preset threshold, determining both the classroom audio text and the classroom courseware content text as the classroom content text.

[0011] By selecting classroom content text and / or classroom courseware content text as the classroom content text required for chapter recognition based on the correlation between classroom audio text and classroom courseware content text, the impact of insufficient ASR recognition on classroom chapter recognition can be compensated for, thereby improving the accuracy of classroom chapter recognition.

[0012] In conjunction with the first aspect, in one possible implementation, the chapter search library is further used to store multiple associated text vectors, which are text vectors representing chapter-related text corresponding to chapter names, and the text vectors represent text in the form of feature vectors; the step of performing similarity matching between the classroom content text and the chapter-related text corresponding to each chapter name in the chapter search library, and determining N candidate chapter names, includes: calculating the similarity between each of the multiple associated text vectors and the text vector of the classroom content text to obtain multiple first similarities, which are the first similarities corresponding to the multiple chapter names, and the first similarities are used to reflect the text similarity between the chapter-related text corresponding to the chapter name and the classroom content text; and determining the N chapter names with the highest first similarities as N candidate chapter names.

[0013] By calculating the similarity between the text vector of the classroom content text and the text vector of the chapter-related text corresponding to the chapter name, multiple candidate chapter names with the highest matching degree with the classroom content text can be determined, which can realize the fine ranking of chapter names in the chapter search library.

[0014] In conjunction with the first aspect, in one possible implementation, the associated text vectors include dense vectors and sparse vectors. The dense vectors are vectors extracted based on a text embedding model to represent the text as a whole, and the sparse vectors are vectors extracted based on the text embedding model to represent the importance of each word in the text. The sparse vectors include weight values ​​corresponding to each word. The step of calculating the similarity between each associated text vector and the text vector of the classroom content text to obtain multiple first similarities includes: calculating the similarity between the target dense vector and the dense vector of the classroom content text to obtain the target chapter... The dense similarity corresponding to the section name is calculated, where the target dense vector is the dense vector of the chapter-related text corresponding to the target section name, and the target section name is any one of the multiple section names; the similarity between the target sparse vector and the sparse vector of the classroom content text is calculated to obtain the sparse similarity corresponding to the target section name, where the target sparse vector is the sparse vector of the chapter-related text corresponding to the target section name; based on the dense similarity and the sparse similarity corresponding to each of the multiple section names, the first similarity corresponding to each section name is determined to obtain the multiple first similarities.

[0015] By using dense and sparse vectors to represent the importance of the text as a whole and each word in the text, respectively, we can provide a more comprehensive and richer feature representation of the classroom content text and the chapter-related text, reducing the impact of data sparsity. By combining the dense and sparse vectors of the classroom content text and the chapter-related text respectively to determine the similarity between the chapter-related text and the classroom content text, we can evaluate the similarity more reasonably and effectively.

[0016] In conjunction with the first aspect, in one possible implementation, calculating the similarity between the target dense vector and the dense vector of the classroom content text to obtain the dense similarity corresponding to the target chapter name includes: calculating the vector inner product between the target dense vector and the dense vector of the classroom content text to obtain the dense similarity corresponding to the target chapter name.

[0017] By calculating the dot product between the dense vectors of chapter-related text and classroom content text, the dense similarity of text can be calculated. This method is fast and can improve the efficiency of similarity calculation.

[0018] In conjunction with the first aspect, in one possible implementation, calculating the similarity between the target sparse vector and the sparse vector of the classroom content text to obtain the sparse similarity corresponding to the target chapter name includes: determining the intersection between a first set of lexical elements and the set of lexical elements in the classroom content text to obtain a first intersection, wherein the first set of lexical elements is the set of lexical elements in the chapter-related text corresponding to the target chapter name, and the set of lexical elements is a set composed of each lexical element in the text, and the first intersection includes at least one lexical element; calculating the product of a first weight value and a second weight value to obtain a sparse weight value corresponding to the first lexical element, wherein the first weight value is the weight value corresponding to the first lexical element in the target sparse vector, the second weight value is the weight value corresponding to the first lexical element in the sparse vector of the classroom content text, and the first lexical element is any lexical element in the first intersection; and summing the sparse weight values ​​corresponding to each lexical element in the first intersection to obtain the sparse similarity corresponding to the target chapter name.

[0019] By calculating the intersection of the word sets of the chapter-related text and the classroom content text, and then multiplying the word sets in the intersection by weights and summing them, the sparse similarity of the texts can be calculated.

[0020] In conjunction with the first aspect, in one possible implementation, the chapter search library is further used to store multiple name text vectors, which are text vectors used to represent chapter names, and the text vectors represent text in the form of feature vectors; the step of matching the N candidate chapter names with the classroom content text based on similarity and determining the classroom chapter name includes: calculating the similarity between the target name text vector and the text vector of the classroom content text to obtain a second similarity corresponding to the target candidate chapter name, where the target name text vector is the text vector of the target candidate chapter name, and the target candidate chapter name is any one of the N candidate chapter names, and the second similarity is used to reflect the text similarity between the chapter name and the classroom content text; based on the second similarity corresponding to each of the N candidate chapter names, the candidate chapter name with the highest second similarity is determined as the classroom chapter name.

[0021] By calculating the similarity between the text vector of the classroom content text and the text vector of the chapter name, the candidate chapter name with the highest matching degree with the classroom content text can be determined, which can realize the recall and reordering of chapter names in the chapter search library.

[0022] In conjunction with the first aspect, in one possible implementation, the name text vector includes a sparse vector, which is a vector extracted based on a text embedding model to represent the importance of each word in the text, and the sparse vector includes a weight value corresponding to each word; calculating the similarity between the target name text vector and the text vector of the classroom content text to obtain a second similarity corresponding to the target candidate chapter name includes: determining the intersection between a second word set and the word set of the classroom content text to obtain a second intersection, wherein the second word set is the word set of the target candidate chapter name, and the word set is a set composed of each word in the text, and the second intersection includes at least one word; calculating the product of a third weight value and a fourth weight value to obtain a sparse weight value corresponding to the second word, wherein the third weight value is the weight value corresponding to the second word in the target name text vector, and the fourth weight value is the weight value corresponding to the second word in the sparse vector of the classroom content text, and the second word is any word in the second intersection; summing the sparse weight values ​​corresponding to each word in the second intersection to obtain the second similarity corresponding to the target candidate chapter name.

[0023] The similarity between chapter titles and classroom content text can be calculated by calculating the intersection of the word sets of chapter titles and classroom content text, multiplying the word sets in the intersection by weights, and then summing the results.

[0024] In conjunction with the first aspect, in one possible implementation, the text embedding model is obtained by fine-tuning a preset embedding model using positive and negative sample data. The preset embedding model is used to convert text into text vectors. The positive sample data includes a preset difficult sample and the first standard courseware text corresponding to the preset difficult sample. The negative sample data includes the preset difficult sample and the second standard courseware text corresponding to the preset difficult sample. The preset difficult sample is audio text or courseware content text that the preset embedding model cannot process. The first standard courseware text is the content text of a first standard courseware, and the chapter name corresponding to the first standard courseware is the same as the actual chapter name corresponding to the preset difficult sample. The second standard courseware text is the content text of a second standard courseware, and the chapter name corresponding to the second standard courseware is different from the actual chapter name corresponding to the preset difficult sample.

[0025] By fine-tuning the preset embedding model used to convert text into text vectors using positive and negative sample data, a text embedding model can be obtained. This allows for targeted adjustments to the text embedding model, enabling the text vectors extracted by the text embedding model to more accurately represent the text in chapter recognition scenarios.

[0026] In conjunction with the first aspect, in one possible implementation, the chapter-related text includes at least two of the following: courseware content text, courseware description text, and courseware content keywords, wherein the courseware description text is text that expresses the courseware content in colloquial language.

[0027] Chapter-related text includes at least two of the following: courseware content text, courseware description text, and courseware content keywords. This enriches the data dimensions of chapter-related text and helps to achieve more accurate chapter name matching.

[0028] In conjunction with the first aspect, in one possible implementation, the courseware description text corresponding to each chapter name in the chapter search library is obtained as follows: A standard courseware screenshot corresponding to the chapter name is obtained; the courseware screenshot prompt information and model prompt text are input into a preset multimodal graphic-text large model for text generation to obtain the courseware description text corresponding to the chapter name. The courseware screenshot prompt information includes the standard courseware screenshot and the screenshot prompt text corresponding to the standard courseware screenshot. The model prompt text is a prompt text that prompts the multimodal graphic-text large model to generate conversational text.

[0029] By inputting the courseware screenshot prompts and model prompts into the multimodal graph-text model, the multimodal graph-text model generates conversational text, which reduces the difference between the courseware description text and the ASR text and helps to better match text.

[0030] In conjunction with the first aspect, in one possible implementation, the courseware content keywords corresponding to each chapter name in the chapter search library are obtained as follows: Hot keywords are extracted from the content of all courseware corresponding to the chapter name to obtain a first keyword set corresponding to the chapter name, where the hot keywords are representative keywords in the courseware content; keywords are extracted from the content of all popular courseware corresponding to the chapter name to obtain a second keyword set corresponding to the chapter name, where the popular courseware is representative; the keywords in the first keyword set and the keywords in the second keyword set are determined as the keywords corresponding to the chapter name.

[0031] By extracting hot keywords from the content of all courseware, coarse-grained keyword extraction can be achieved; by extracting keywords from the content of popular courseware, fine-grained keyword extraction can be achieved; using the keywords obtained from coarse-grained and fine-grained extraction as the keywords corresponding to the chapter titles can enrich the keywords in the courseware content, thereby enriching the content of the chapter-related text.

[0032] Secondly, a classroom chapter recognition device is provided, comprising:

[0033] The text acquisition module is used to acquire classroom content text, which includes classroom audio text and / or classroom courseware content text corresponding to the classroom audio text. The classroom audio text is obtained by automatic speech recognition of the classroom audio.

[0034] The first matching module is used to perform similarity matching between the classroom content text and the chapter-related text corresponding to each chapter name in the chapter search library, and to determine N candidate chapter names. The chapter search library is used to store multiple chapter names, and the chapter-related text is text associated with the chapter name to introduce the chapter knowledge. The chapter-related text corresponding to the N candidate chapter names has the highest matching degree with the classroom content text, and N is a positive integer greater than or equal to 2.

[0035] The second matching module is used to perform similarity matching between the N candidate chapter names and the classroom content text, and to determine the classroom chapter name, which is the candidate chapter name with the highest matching degree with the classroom content text.

[0036] Thirdly, a computer device is provided, including a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, wherein when the processor executes the one or more computer programs, the computer device enables the classroom chapter recognition method of the first aspect described above.

[0037] Fourthly, a computer-readable storage medium is provided, which stores a computer program, the computer program including program instructions, which, when executed by a processor, cause the processor to perform the classroom chapter recognition method of the first aspect.

[0038] This application can achieve the following technical effects: by first refining the chapter names based on the chapter-related text corresponding to the chapter names, and then retrieving and re-ranking the chapter names based on the chapter names, the classroom chapter names corresponding to the classroom content can be determined, which can ensure the accuracy of classroom chapter identification. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a classroom chapter recognition method provided in an embodiment of this application;

[0041] Figure 2 A schematic diagram illustrating the process of obtaining classroom content text provided in an embodiment of this application;

[0042] Figure 3 A schematic diagram of a text embedding model, dense vectors, and sparse vectors provided for embodiments of this application;

[0043] Figure 4 This is a schematic diagram of the structure of a classroom chapter recognition device provided in an embodiment of this application;

[0044] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0046] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0047] The technical solution of this application is applicable to classroom observation scenarios, and can be used to identify classroom chapters in classroom observation scenarios.

[0048] In a feasible classroom chapter identification scheme, each chapter has corresponding courseware. Beforehand, the content of different courseware corresponding to a chapter is statistically analyzed to identify frequently occurring keywords. These keywords are used as features for each chapter. Features are then determined for all chapters in the same way, forming a chapter feature database. The BM25 algorithm is used to calculate the similarity between the ASR text identified in the classroom observation scenario and the keywords in the chapter feature database. The chapter corresponding to the keyword with the highest similarity is then identified as the classroom chapter.

[0049] Since ASR text is not always accurate, the accuracy of classroom chapters identified using the above-mentioned classroom chapter recognition scheme is not high.

[0050] In view of this, this application proposes a novel chapter recognition scheme. By pre-constructing multi-dimensional chapter-related texts for all chapters and forming a chapter search library, during classroom chapter recognition, the classroom content text is first matched with the chapter-related texts corresponding to each chapter name in the chapter search library for similarity, determining N candidate chapter names. The chapter-related texts corresponding to these N candidate chapter names have the highest matching degree with the classroom content text, enabling fine-grained ranking of chapter names in the chapter search library and identifying the multiple chapter names most likely to correspond to the classroom content. Next, the N candidate chapter names are matched with the classroom content text for similarity, and the candidate chapter name with the highest matching degree is determined as the classroom chapter name. This enables recall and re-ranking of chapter names, ultimately determining the classroom chapter name corresponding to the classroom content. By first fine-grained ranking of chapter names based on their corresponding chapter-related texts and then recalling and re-ranking them based on the chapter names themselves, the accuracy of classroom chapter recognition can be guaranteed.

[0051] The technical solution of this application is described in detail below. The technical solution of this application can be applied to computer equipment, including but not limited to laptops, desktop computers, servers or server clusters.

[0052] See Figure 1 , Figure 1 A flowchart illustrating a classroom chapter recognition method provided in this application embodiment is shown below. Figure 1 As shown, the method includes the following steps:

[0053] S101, Obtain the text of the classroom content.

[0054] Here, classroom content text refers to the text used to represent the teaching content in the classroom.

[0055] Classroom content text includes classroom audio text and / or the corresponding classroom presentation slides text. Classroom audio text represents the teaching content delivered orally in class, obtained through automatic speech recognition of the classroom audio. Presentation slides text represents the teaching content presented in presentation slides in class, obtained through optical character recognition (OCR) of the presentation slides. The classroom presentation slides text corresponding to the classroom audio text refers to the presentation slides text belonging to the same classroom as the classroom audio text.

[0056] In one feasible implementation, it can be achieved by... Figure 2 The process steps shown are for obtaining classroom content text, including the following steps A1-A7:

[0057] A1. Obtain the classroom audio text and the corresponding classroom courseware content text.

[0058] This allows for the acquisition of classroom audio and courseware belonging to the same class, the use of ASR to obtain classroom audio text, and the use of OCR to recognize the content of courseware to obtain classroom courseware content text.

[0059] A2. Calculate the correlation between the classroom audio text and the corresponding classroom courseware content text.

[0060] The relevance between classroom audio text and the corresponding classroom courseware content text can be calculated using any text relevance calculation method. Text relevance calculation methods include, but are not limited to, the term frequency-inverse document frequency (TF-IDF) algorithm, the BM25 algorithm, word weighting algorithms, and so on.

[0061] In some specific embodiments, the text vectors of the classroom audio text and the corresponding classroom courseware content text can be extracted. The similarity between these two text vectors is then calculated, and this similarity is determined as the correlation between the two texts. A text embedding model can be used to extract the text vectors of the classroom audio text and the corresponding classroom courseware content text. This model converts text into vectors and can be any type of embedding model, such as bge-m3. The similarity between the text vectors of the classroom audio text and the corresponding classroom courseware content text can be calculated using Euclidean distance or cosine similarity, etc.

[0062] This application does not limit the specific method for calculating the correlation between classroom audio text and the corresponding classroom courseware content text.

[0063] A3. Determine whether the correlation between the classroom audio text and the corresponding classroom courseware content text is less than or equal to the first preset threshold.

[0064] Here, the first preset threshold is a preset threshold used to determine whether the classroom audio text and the classroom courseware content text are related. The first preset threshold can be, for example, 0.5.

[0065] If the correlation between the classroom audio text and the corresponding classroom courseware content text is less than or equal to the first preset threshold, it means that the corresponding classroom courseware content text is not related to the classroom audio text, and proceed to step A4; if the correlation between the classroom audio text and the corresponding classroom courseware content text is greater than the first preset threshold, it means that the corresponding classroom courseware content text is related to the classroom audio text, and proceed to step A5.

[0066] A4. Classroom audio texts are designated as classroom content texts.

[0067] A5. Determine whether the correlation between the classroom audio text and the corresponding classroom courseware content text is greater than or equal to the second preset threshold.

[0068] Here, the second preset threshold is a preset threshold used to determine whether the classroom audio text and the classroom courseware content text are highly correlated. The second preset threshold is greater than the first preset threshold. For example, the second preset threshold can be 0.8.

[0069] If the correlation between the classroom audio text and the corresponding classroom courseware content text is greater than or equal to the second preset threshold, it indicates that the classroom courseware content text corresponding to the classroom audio text is highly correlated with the classroom audio text, and step A6 is executed; if the correlation between the classroom audio text and the corresponding classroom courseware content text is less than the second preset threshold, it indicates that the correlation between the classroom courseware content text corresponding to the classroom audio text and the classroom audio text is average, and step A7 is executed.

[0070] A6. The text of the classroom courseware corresponding to the classroom audio text shall be identified as the classroom content text.

[0071] A7. The classroom audio text and the corresponding classroom courseware content text are identified as the classroom content text.

[0072] In steps A1-A7 above, by selecting classroom content text and / or classroom courseware content text as the classroom content text required for chapter recognition based on the correlation between classroom audio text and classroom courseware content text, the impact of insufficient ASR recognition on classroom chapter recognition can be compensated, and the accuracy of classroom chapter recognition can be improved.

[0073] S102, perform similarity matching between the classroom content text and the chapter-related text corresponding to each chapter name in the chapter search database, and determine N candidate chapter names.

[0074] Here, the chapter search library is used to store multiple chapter names.

[0075] The chapter-related text corresponding to the chapter title is text associated with the chapter title that introduces the chapter's information. Details regarding the content and acquisition methods of this chapter-related text will be provided later; it will not be elaborated upon here.

[0076] The N candidate chapter names are those whose associated text matches the classroom content text the most closely. N is a positive integer greater than or equal to 2. In other words, the N candidate chapter names are the chapter names corresponding to the N chapter-associated texts that match the classroom content text the most closely.

[0077] This involves using any text similarity matching algorithm to match the chapter-related texts corresponding to each chapter name in the chapter search database with the classroom content text. The goal is to identify the N chapter-related texts with the highest matching degree, and then determine the N candidate chapter names corresponding to these N texts. Text similarity matching algorithms include, but are not limited to, statistical text similarity matching algorithms, string-based text similarity matching algorithms (such as the N-Gram algorithm), and embedding model-based text similarity matching algorithms, etc.

[0078] S103, perform similarity matching between the N candidate chapter names and the classroom content text, and determine the classroom chapter names.

[0079] Here, the classroom chapter name is the candidate chapter name with the highest matching degree among N candidate chapter names.

[0080] This process involves using any text similarity matching algorithm to match each of the N candidate chapter names with the classroom content text, identifying the candidate chapter name with the highest matching score, and then determining that candidate chapter name as the classroom chapter name. Text similarity matching algorithms include, but are not limited to, statistical text similarity matching algorithms, string-based text similarity matching algorithms, and embedding model-based text similarity matching algorithms, etc.

[0081] In the above Figure 1 In the corresponding technical solution, after obtaining the classroom content text, the text is first matched with the associated text corresponding to each chapter name in the chapter search library to determine N candidate chapter names. Since the associated text is text used to introduce the chapter knowledge associated with the chapter name, the associated text corresponding to the N candidate chapter names has the highest matching degree with the classroom content text, which can realize the fine ranking of chapter names in the chapter search library and determine the multiple chapter names most likely to correspond to the classroom content. Then, the N candidate chapter names are matched with the classroom content text to determine the classroom chapter name. The classroom chapter name is the candidate chapter name with the highest matching degree with the classroom content text, which can realize the recall and re-ranking of chapter names, thereby determining the final classroom chapter name corresponding to the classroom content. By first refining the chapter names according to the associated text corresponding to the chapter names and then re-ranking the chapter names according to the chapter names to determine the classroom chapter name corresponding to the classroom content, the accuracy of classroom chapter identification can be guaranteed.

[0082] In some embodiments, the chapter-related text corresponding to a chapter name includes at least two of the following: the courseware content text corresponding to the chapter name, the courseware description text corresponding to the chapter name, and the courseware content keywords corresponding to the chapter name. The courseware content text corresponding to the chapter name is text representing the content of the courseware corresponding to the chapter name; the courseware description text corresponding to the chapter name is text expressing the courseware content in colloquial language; and the courseware content keywords corresponding to the chapter name are text representing the content keywords of the courseware corresponding to the chapter name. The inclusion of at least two of these elements in the chapter-related text enriches the data dimensions of the chapter-related text and helps achieve more accurate chapter name matching.

[0083] The courseware content text corresponding to each chapter name in the chapter search library can be obtained by OCR extraction of the courseware content corresponding to the chapter name. By pre-collecting the courseware corresponding to each chapter name, a courseware set corresponding to each chapter name is obtained. Each courseware set corresponding to each chapter name includes one or more courseware. The content of the courseware in the courseware set corresponding to each chapter name is extracted by OCR, thereby obtaining the courseware content text corresponding to each chapter name in the chapter search library.

[0084] The courseware description text corresponding to each chapter name in the chapter search database can be obtained in the following way:

[0085] (1) Obtain the standard courseware screenshot corresponding to the chapter name; the standard courseware screenshot corresponding to the chapter name refers to the screenshot of the representative courseware set for the chapter name. The screenshot of the representative courseware is, for example, the screenshot of the courseware with the most downloads, the screenshot of the courseware with the most likes, or the screenshot of the courseware with the most usage frequency, etc.

[0086] (2) Input the courseware screenshot prompt information and model prompt text into the preset multimodal graphic and text big model for text generation to obtain the courseware description text corresponding to the chapter name; the courseware screenshot prompt information includes standard courseware screenshots and screenshot prompt text corresponding to standard courseware screenshots. The screenshot prompt text is a prompt text used to prompt the multimodal graphic and text big model to generate text and to describe the textbook version, subject type, and other information of the standard courseware screenshot. The model prompt text is a prompt text that prompts the multimodal graphic and text big model to generate conversational text.

[0087] For example, the screenshot prompt text could be: "Please recreate the classroom based on the uploaded image (referring to the standard courseware screenshot in this application). The relevant information of the image is: xxx." The model prompt text could be: "Please combine the new curriculum standards to carefully understand the content on the courseware image, and verbally describe the content of the courseware image. Do not simply repeat the text content on the image; you need to enrich the content on the image, and the content you describe should appropriately interact with students to enhance their learning interest."

[0088] The preset multimodal image-text model is an artificial intelligence model that combines natural language processing and computer vision technologies, capable of understanding and processing various types of text and images, and generating corresponding image content or text descriptions based on user needs. An example of a preset multimodal image-text model is GPT-4o.

[0089] By pre-collecting standard courseware screenshots corresponding to each chapter name and pre-determining the courseware screenshot prompts for each chapter name, and then inputting the courseware screenshot prompts and model prompt text for each chapter name into the multimodal graphic-text big model for text generation, the courseware description text corresponding to the chapter name can be obtained, thereby obtaining the courseware description text corresponding to each chapter name in the chapter search library.

[0090] By inputting the courseware screenshot prompts and model prompts into the multimodal graph-text model, the multimodal graph-text model generates conversational text, which reduces the difference between the courseware description text and the ASR text and helps to better match text.

[0091] The keywords for the courseware content corresponding to each chapter name in the chapter search database can be obtained in the following way:

[0092] (1) Extract hot keywords from the content of all courseware corresponding to the chapter name to obtain the first keyword set corresponding to the chapter name. The first keyword set includes all the extracted hot keywords, which are representative keywords in the courseware content.

[0093] Specifically, the TF-IDF probability can be calculated for the keywords appearing in the content of all courseware corresponding to the chapter name based on the TF-IDF algorithm to obtain the word frequency probability of each keyword. Based on the word frequency probability of each keyword, the top O keywords with word frequency probabilities greater than the preset probability or the highest probability are taken as hot keywords, thus obtaining the first keyword set, where O is a positive integer greater than 1.

[0094] (2) Extract keywords from the content of all popular courseware corresponding to the chapter name to obtain the second keyword set corresponding to the chapter name. The second keyword set includes all extracted keywords. Popular courseware refers to representative courseware. Popular courseware may include the courseware with the most downloads and / or the courseware with the most likes and / or the courseware with the most usage frequency, etc.

[0095] Keywords can be extracted from the courseware content using any keyword extraction algorithm. For example, GPT-4o can be used to extract keywords appearing in the courseware content.

[0096] (3) After extracting the first keyword set and the second keyword set, the keywords in the first keyword set and the keywords in the second keyword set are determined as the keywords corresponding to the chapter names.

[0097] By pre-collecting all courseware corresponding to each chapter name and extracting hot keywords from the content of all courseware corresponding to each chapter name, a first set of keywords corresponding to each chapter name is obtained. Furthermore, by extracting keywords from the content of all popular courseware corresponding to each chapter name, a second set of keywords is obtained. The keywords from the first set and the second set of keywords corresponding to each chapter name are then used to determine the keywords for each chapter name, thus obtaining the courseware content keywords for each chapter name in the chapter search database.

[0098] By extracting hot keywords from the content of all courseware, coarse-grained keyword extraction can be achieved; by extracting keywords from the content of popular courseware, fine-grained keyword extraction can be achieved; using the keywords obtained from coarse-grained and fine-grained extraction as the keywords corresponding to the chapter titles can enrich the keywords in the courseware content, thereby enriching the content of the chapter-related text.

[0099] After obtaining the chapter-related texts corresponding to each chapter name in the chapter search library, the chapter-related texts corresponding to each chapter name can be saved to the chapter search library, and / or, text vectors of the chapter-related texts corresponding to each chapter name can be extracted and saved to the chapter search library, and / or, tokens contained in the chapter-related texts corresponding to each chapter name can be extracted to obtain token sets of the chapter-related texts corresponding to each chapter name, and token sets of the chapter-related texts corresponding to each chapter name can be saved to the chapter search library. That is, in addition to storing multiple chapter names, the chapter search library is also used to store multiple chapter-related texts, and / or, to store multiple related text vectors, and / or, to store multiple token sets. Multiple chapter-related texts are chapter-related texts corresponding to multiple chapter names; multiple related text vectors are related text vectors corresponding to multiple chapter names, with one related text vector corresponding to one chapter name, and the related text vector is a text vector used to represent the chapter-related text corresponding to a chapter name; multiple token sets are token sets corresponding to multiple chapter names, and the token set is a set composed of tokens contained in the chapter-related texts corresponding to chapter names.

[0100] Text vectors represent text in the form of feature vectors. A text embedding model extracts feature vectors from the chapter-related text corresponding to each chapter name, resulting in text vectors for each chapter-related text. These text vectors form multiple related text vectors in the chapter search library. The text embedding model is used to convert text into feature vectors. Different text embedding models used to extract the feature vectors will result in different meanings and representations of the text vectors. For each chapter-related text in multiple chapter-related texts, the same text embedding model is used to extract feature vectors from each chapter-related text, resulting in multiple related text vectors. This same text embedding model could be, for example, bge-m3.

[0101] A lexical unit is the smallest semantic unit obtained by segmenting text. A lexical unit contains one or more English words, or one Chinese character. By extracting lexical units from the chapter-related text corresponding to each chapter name using a word segmentation model, a lexical unit set for each chapter-related text is obtained, which includes each lexical unit obtained from word segmentation.

[0102] Specifically, when the chapter-related text corresponding to a chapter name includes at least two of the following: the courseware content text corresponding to the chapter name, the courseware description text corresponding to the chapter name, and the courseware content keywords corresponding to the chapter name, the text vector of each chapter-related text corresponding to a chapter name is obtained by fusing at least two of the following three elements and then extracting them through a text embedding model: Alternatively, the text vector of each chapter-related text corresponding to a chapter name is obtained by extracting feature vectors from at least two of the following three elements separately through a text embedding model, and then fusing the extracted feature vectors. The word set of each chapter-related text corresponding to a chapter name is a set of words obtained by fusing at least two of the following three elements and then segmenting them into words.

[0103] For example, suppose the courseware content text a1, the courseware description text a2, and the courseware content keyword a3 are all chapter-related texts corresponding to the chapter name na. We can combine these texts into a fused text a = {a1, a2, a3}. Then, using a text embedding model, we can extract the text vector Ta of the fused text a. This text vector Ta can then be used as the text vector of the chapter-related text corresponding to the chapter name na (i.e., the associated text vector corresponding to the chapter name na). Alternatively, we can extract the text vectors of the courseware content text a1, the courseware description text a2, and the courseware content keyword a3 separately using a text embedding model, obtaining text vectors T1, T2, and T3. Concatenating these text vectors yields text vector Ta, which can then be used as the text vector of the chapter-related text corresponding to the chapter name na (i.e., the associated text vector corresponding to the chapter name na). The courseware content text a1, the courseware description text a2, and the courseware content keywords a3 are segmented into words to obtain word set j1, word set j2, and word set j3. The word set J = {j1, j2, j3} is used as the word set corresponding to the chapter name na.

[0104] In some embodiments, the similarity of the classroom content text with the associated text corresponding to each chapter name in the chapter search library can be performed based on text vector matching to determine N candidate chapter names. The chapter search library is also used to store multiple associated text vectors. For the definition of multiple associated text vectors, please refer to the foregoing description. The N candidate chapter names can be determined through the following steps B1-B2:

[0105] B1. Calculate the similarity between each associated text vector in the chapter search library and the text vector of the classroom content text to obtain multiple first similarity scores.

[0106] Here, multiple first similarities refer to the first similarities corresponding to multiple chapter names. The first similarity is used to reflect the text similarity between the chapter-related text corresponding to the chapter name and the classroom content text.

[0107] The text vectors of the classroom content text are obtained by extracting feature vectors from the classroom content text using a text embedding model. The text embedding model for the extracted text vectors of the classroom content text is the same as the text embedding model for the extracted associated text vectors.

[0108] The text vectors used to calculate the first similarity of the classroom content text are represented in the same way as the associated text vectors used to calculate the first similarity.

[0109] In one embodiment, the associated text vector used to calculate the first similarity includes a dense vector and a sparse vector, and correspondingly, the text vector of the classroom content text used to calculate the first similarity also includes a dense vector and a sparse vector.

[0110] The dense vector is a vector extracted based on the text embedding model to represent the text as a whole, while the sparse vector is a vector extracted based on the text embedding model to represent the importance of each word in the text. The sparse vector includes the weight value corresponding to each word in the text. (See reference...) Figure 3 , Figure 3 The diagram illustrates a text embedding model and the concepts of dense and sparse vectors. After arranging the words in the set corresponding to the text in order to obtain a text word sequence, a sequence start word [CLS] and a sequence end word [SEP] are added to the beginning and end of the text word sequence, respectively, forming the input word sequence. This input word sequence is then fed into the text embedding model, which converts each word in the input word sequence into a feature vector of a preset dimension. Each word in the input word sequence corresponds to a feature vector. The feature vector corresponding to the sequence start word [CLS] is normalized by a dense embedding layer to obtain a dense vector. The feature vectors of the words in the set corresponding to the text are transformed by a linear layer to obtain the weights of each word in the text. The weights of each word in the text form a sparse vector.

[0111] The similarity between each associated text vector in the chapter search database and the text vector of the classroom content text can be calculated through the following steps a1-a3 to obtain multiple first similarity scores:

[0112] a1. Calculate the similarity between the target dense vector and the dense vector of the classroom content text to obtain the dense similarity corresponding to the target chapter name.

[0113] Here, the target dense vector is the dense vector of the chapter-related text corresponding to the target chapter name, and the target chapter name is any one of the multiple chapter names in the chapter search library.

[0114] The target dense vector is the dense vector output by the text embedding model after the chapter-related text corresponding to the target chapter name is input into the text embedding model. The dense vector of the classroom content text is the dense vector output by the text embedding model after the classroom content text is input into the text embedding model. The principle of the text embedding model outputting dense vectors can be referred to the above. Figure 3 The relevant description.

[0115] In one specific implementation, the dot product between the target dense vector and the dense vector of the classroom content text can be calculated to obtain the dense similarity corresponding to the target chapter name.

[0116] The formula for calculating density similarity is:

[0117]

[0118] sc i X represents the dense similarity of the i-th chapter name in the chapter search database, where n represents the vector dimension of the dense vector. ik Y represents the k-th vector value in the dense vector corresponding to the i-th chapter name. k This represents the k-th vector value in the dense vector of the classroom content text.

[0119] By calculating the dot product between the dense vectors of chapter-related text and classroom content text, the dense similarity of text can be calculated. This method is fast and can improve the efficiency of similarity calculation.

[0120] Alternatively, the density similarity corresponding to the target chapter name can be obtained by calculating the cosine similarity, Euclidean distance, etc., between the target dense vector and the dense vector of the classroom content text. This application does not impose any restrictions on this.

[0121] For each chapter name in the chapter search library, the dense similarity corresponding to the chapter name can be calculated in the manner described in step a1, thereby obtaining the dense similarity corresponding to each chapter name among multiple chapter names in the chapter search library.

[0122] a2. Calculate the similarity between the target sparse vector and the sparse vector of the classroom content text to obtain the sparse similarity corresponding to the target chapter name.

[0123] Here, the target sparse vector is the sparse vector of the chapter-related text corresponding to the target chapter name.

[0124] The target sparse vector is the sparse vector output by the text embedding model after the chapter-related text corresponding to the target chapter name is input into the text embedding model. The sparse vector of the classroom content text is the sparse vector output by the text embedding model after the classroom content text is input into the text embedding model. The principle of the text embedding model outputting sparse vectors can be referred to the above. Figure 3 The relevant description.

[0125] In one specific implementation, the sparse similarity corresponding to the target chapter name can be obtained through the following steps a21-a23:

[0126] a21. Determine the intersection between the first set of lexical units and the set of lexical units in the classroom content text to obtain the first intersection.

[0127] Here, the first word set is the word set of the chapter-related text corresponding to the target chapter name. The first word set includes all word sets obtained by segmenting the chapter-related text corresponding to the target chapter name. The word set of the classroom content text includes all word sets obtained by segmenting the classroom content text. For the meaning of word sets, please refer to the above introduction.

[0128] The first intersection includes at least one word element. The word elements in the first intersection belong to both the first word element set and the word element set of the classroom content text.

[0129] a22. Calculate the product of the first weight value and the second weight value to obtain the sparse weight value corresponding to the first word.

[0130] Here, the first weight value is the weight value of the first word element in the target sparse vector, and the second weight value is the weight value of the first word element in the sparse vector of the classroom content text. The first word element is any word element in the first intersection.

[0131] a23. Sum the sparse weight values ​​corresponding to each word in the first intersection to obtain the sparse similarity corresponding to the target chapter name.

[0132] The formula for calculating sparse similarity is:

[0133]

[0134] ss i w1 represents the sparse similarity of the i-th chapter name in the chapter search database, m represents the number of tokens in the first intersection, and w1 represents the number of tokens in the first intersection. ij w2 represents the weight value of the j-th word in the sparse vector corresponding to the i-th chapter name in the first intersection set. jThis represents the weight value of the j-th word in the first intersection set within the sparse vector of the classroom content text.

[0135] In steps a21-a23 above, the sparse similarity of the texts can be calculated by calculating the intersection of the word set of the chapter-related text and the classroom content text, multiplying the word sets in the intersection by weights, and then summing them.

[0136] For each chapter name in the chapter search library, the sparse similarity corresponding to the chapter name can be calculated in the manner described in step a2, thereby obtaining the sparse similarity corresponding to each chapter name among multiple chapter names in the chapter search library.

[0137] a3. Based on the dense similarity and sparse similarity of each chapter name in the multiple chapter names, determine the first similarity of each chapter name and obtain multiple first similarities.

[0138] In one specific implementation, the dense similarity and sparse similarity of each chapter name in multiple chapter names can be weighted and summed to obtain the first similarity of each chapter name, thus obtaining multiple first similarities.

[0139] The formula for calculating the first similarity is: S1 i =W1*sc i +W2*ss i S1 i Let represent the first similarity score corresponding to the i-th chapter name in the chapter search database, W1 represent the weight of dense similarity, and sc i Let Wi represent the dense similarity corresponding to the i-th chapter name in the chapter search database, W2 represent the weight of the sparse similarity, and ss represent the sparse similarity. i This represents the sparse similarity corresponding to the i-th chapter name in the chapter search database.

[0140] In steps a1-a3 above, dense vectors and sparse vectors are used to represent the importance of the text as a whole and each word in the text, respectively. This enables a more comprehensive and richer feature representation of the classroom content text and the chapter-related text, reducing the impact of data sparsity. Combining the dense vectors and sparse vectors of the classroom content text and the chapter-related text respectively to determine the similarity between the chapter-related text and the classroom content text allows for a more reasonable and effective evaluation of similarity.

[0141] It is understood that, in addition to being represented as dense vectors and sparse vectors, the associated text vectors used to calculate the first similarity and the text vectors of the classroom content texts used to calculate the first similarity can also be represented as other vector forms, such as dense vectors or sparse vectors; there can also be other ways to calculate the first similarity, and this application does not impose any restrictions on this.

[0142] B2. The N chapter names with the highest first similarity are determined as N candidate chapter names.

[0143] Here, the N chapter names with the highest first similarity refer to the N chapter names with the highest first similarity in the chapter search database. Taking N=10 as an example, after calculating the first similarity of each of the multiple chapter names in the chapter search database, the 10 chapter names with the highest first similarity in the chapter search database can be determined as 10 candidate chapter names.

[0144] In steps B1-B2 above, the similarity between the text vector of the classroom content text and the text vector of the chapter-related text corresponding to the chapter name is calculated to determine the multiple candidate chapter names with the highest matching degree with the classroom content text, which can realize the fine ranking of chapter names in the chapter search library.

[0145] In some embodiments, N candidate chapter names can be matched with the classroom content text based on text vector matching to determine the classroom chapter name. The chapter search library also stores multiple name text vectors, which are name text vectors corresponding to multiple chapter names. Each chapter name corresponds to one name text vector, which is a text vector used to represent the chapter name. The name text vectors are obtained by extracting text vectors from the chapter names. Feature vectors of each chapter name in the chapter search library can be extracted using a text embedding model to obtain multiple name text vectors. The classroom chapter name can be determined through the following steps C1-C2:

[0146] C1. Calculate the similarity between the text vector of the target name and the text vector of the classroom content to obtain the second similarity corresponding to the target candidate chapter name.

[0147] Here, the target name text vector is the text vector of the target candidate chapter name, and the target candidate chapter name is any one of the N candidate chapter names. The second similarity is used to reflect the text similarity between the chapter name and the classroom content.

[0148] The target name text vector is obtained by extracting the feature vector of the target candidate chapter name through a text embedding model.

[0149] The representation of the name text vector used to calculate the second similarity is the same as the representation of the text vector of the classroom content text used to calculate the second similarity.

[0150] In one embodiment, the name text vector used to calculate the second similarity includes a sparse vector, and correspondingly, the text vector of the classroom content text used to calculate the second similarity also includes a sparse vector. For a definition of a sparse vector, please refer to the foregoing. Figure 3 The corresponding description. The similarity between the text vector of the target name and the text vector of the classroom content can be calculated using the following steps b1-b3 to obtain the second similarity corresponding to the target candidate chapter name:

[0151] b1. Determine the intersection between the second set of lexical units and the set of lexical units in the classroom content text to obtain the second intersection.

[0152] Here, the second lexical set is the lexical set of the target candidate chapter name. The second lexical set includes all lexical units obtained by segmenting the target candidate chapter name. For the meaning of the lexical set of the classroom content text, please refer to the description of step a21 above.

[0153] The second intersection includes at least one word element. The word elements in the second intersection belong to both the second word element set and the word element set of the classroom content text.

[0154] b2. Calculate the product of the third weight value and the fourth weight value to obtain the sparse weight value corresponding to the second word.

[0155] Here, the third weight value is the weight value of the second word element in the target name text vector, and the fourth weight value is the weight value of the second word element in the sparse vector of the classroom content text. The second word element is any word element in the second intersection.

[0156] b3. Sum the sparse weight values ​​corresponding to each word in the second intersection to obtain the second similarity corresponding to the target candidate chapter name.

[0157] Here, the formula for calculating the second similarity is:

[0158]

[0159] S2 u w3 represents the second similarity among the u-th candidate chapter names out of N candidate chapter names, M represents the number of lexical units in the second intersection, and w3 represents the number of lexical units in the second intersection. uj w4 represents the weight value of the text vector corresponding to the j-th word in the second intersection set and the u-th candidate chapter name. j This represents the weight value of the j-th word in the second intersection set within the sparse vector of the classroom content text.

[0160] In steps b1-b3 above, the similarity between the chapter title and the classroom content text can be calculated by calculating the intersection of the word set of the chapter title and the word set of the classroom content text, multiplying the weights of the words in the intersection, and then summing them.

[0161] For each of the N candidate chapter names, the second similarity can be calculated according to the steps b1-b3 above, thereby obtaining the second similarity of each candidate chapter name among the N candidate chapter names.

[0162] It is understood that, in addition to the sparse vector form of the name text vector used to calculate the second similarity and the sparse vector form of the classroom content text used to calculate the second similarity, other vector forms are also possible, such as dense vector forms, or dense and sparse vector forms; there are also other ways to calculate the second similarity, and this application does not impose any restrictions on this.

[0163] C2. Based on the second similarity of each candidate chapter name among the N candidate chapter names, the candidate chapter name with the highest second similarity is determined as the classroom chapter name.

[0164] In steps C1-C2 above, the candidate chapter name with the highest matching degree with the classroom content text is determined by calculating the similarity between the text vector of the classroom content text and the text vector of the chapter name, which can realize the recall and reordering of chapter names in the chapter search library.

[0165] In some embodiments, before extracting feature vectors from the aforementioned chapter-related text, classroom content text, and chapter names using a text embedding model to obtain the related text vectors, classroom content text text vectors, and name text vectors, the preset embedding model can be fine-tuned using positive and negative sample data to obtain the aforementioned text embedding model. The preset embedding model is used to convert text into text vectors, and a schematic diagram of the preset embedding model can be shown below. Figure 3As shown, the preset embedding model can output dense and sparse vectors based on the input sequence corresponding to the text. Positive sample data includes preset difficult samples and the first standard courseware text corresponding to the preset difficult samples. Negative sample data includes preset difficult samples and the second standard courseware text corresponding to the preset difficult samples. Preset difficult samples are audio text or courseware content text that the preset embedding model cannot process. The first standard courseware text is the content text of the first standard courseware, and the chapter name corresponding to the first standard courseware is the same as the real chapter name corresponding to the preset difficult sample. The second standard courseware text is the content text of the second standard courseware, and the chapter name corresponding to the second standard courseware is different from the real chapter name corresponding to the preset difficult sample. Standard courseware refers to representative courseware, such as the courseware with the most downloads and / or the courseware with the most likes and / or the courseware with the highest usage frequency. Fine-tuning the preset embedding model using positive and negative sample data refers to adjusting the parameters of the preset embedding model to increase the similarity between two text vectors in the first comparison vector combination output by the adjusted preset embedding model, and to decrease the similarity between two text vectors in the second comparison vector combination output by the adjusted preset embedding model. The first comparison vector combination includes text vectors obtained by extracting feature vectors from preset difficult samples in the positive sample data using the preset embedding model, and text vectors obtained by extracting feature vectors from the first standard courseware text in the positive sample data using the preset embedding model. The second comparison vector combination includes text vectors obtained by extracting feature vectors from preset difficult samples in the negative sample data using the preset embedding model, and text vectors obtained by extracting feature vectors from the second standard courseware text in the negative sample data using the preset embedding model.

[0166] By fine-tuning the preset embedding model used to convert text into text vectors using positive and negative sample data, a text embedding model can be obtained. This allows for targeted adjustments to the text embedding model, enabling the text vectors extracted by the text embedding model to more accurately represent the text in chapter recognition scenarios.

[0167] The method of this application has been described above; the apparatus of this application will be described below.

[0168] See Figure 4 , Figure 4 This is a schematic diagram of the structure of a classroom chapter recognition device provided in an embodiment of this application, such as... Figure 4 As shown, the classroom chapter recognition device 20 includes:

[0169] The text acquisition module 201 is used to acquire classroom content text, which includes classroom audio text and / or classroom courseware content text corresponding to the classroom audio text. The classroom audio text is obtained by automatic speech recognition of the classroom audio.

[0170] The first matching module 202 is used to perform similarity matching between the classroom content text and the chapter-related text corresponding to each chapter name in the chapter search library, and determine N candidate chapter names. The chapter search library is used to store multiple chapter names. The chapter-related text is text associated with the chapter name and used to introduce the chapter knowledge. The chapter-related text corresponding to the N candidate chapter names has the highest matching degree with the classroom content text. N is a positive integer greater than or equal to 2.

[0171] The second matching module 203 is used to perform similarity matching between the N candidate chapter names and the classroom content text, and to determine the classroom chapter name, wherein the classroom chapter name is the candidate chapter name with the highest matching degree with the classroom content text.

[0172] It should be noted that the classroom chapter recognition device 20 described above can execute the classroom chapter recognition method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in the embodiments of the classroom chapter recognition device 20 can be found in the classroom chapter recognition method provided in the embodiments of this application.

[0173] See Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device 30 provided in an embodiment of this application. The computer device 30 includes a processor 301 and a memory 302. The memory 302 is connected to the processor 301, for example, via a bus.

[0174] Processor 301 is configured to support the computer device 30 in performing the corresponding functions in the methods described in the above method embodiments. Processor 301 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0175] Memory 302 is used to store program code, etc. Memory 302 may include volatile memory (VM), such as random access memory (RAM); memory 302 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 302 may also include combinations of the above types of memory.

[0176] The memory 302 is used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the classroom chapter recognition method in the embodiments of this application. The processor executes various functional applications and data processing of the classroom chapter recognition method by running the non-volatile software programs, instructions, and modules stored in the memory, thereby realizing the functions of the classroom chapter recognition method provided in the above method embodiments.

[0177] The memory 302 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the classroom chapter recognition device. In some embodiments, the memory may include memory remotely located relative to the processor, which can be connected to the classroom chapter recognition device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0178] The one or more modules are stored in the memory. When executed by the one or more processors, they perform the classroom chapter recognition method in any of the above method embodiments. For example, they perform the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.

[0179] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method described in the foregoing embodiments.

[0180] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0181] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for identifying classroom chapters, characterized in that, include: Obtain classroom content text, which includes classroom audio text and / or classroom courseware content text corresponding to the classroom audio text. The classroom audio text is obtained by automatic speech recognition of the classroom audio. The similarity of the classroom content text with the chapter-related text corresponding to each chapter name in the chapter search library is performed to determine N candidate chapter names. The chapter search library is used to store multiple chapter names. The chapter-related text is the text associated with the chapter name and used to introduce the chapter knowledge. The chapter-related text corresponding to the N candidate chapter names has the highest matching degree with the classroom content text. N is a positive integer greater than or equal to 2. The N candidate chapter names are matched with the classroom content text for similarity, and the classroom chapter name is determined. The classroom chapter name is the candidate chapter name with the highest matching degree with the classroom content text.

2. The method according to claim 1, characterized in that, The acquisition of classroom content text includes: Obtain the classroom audio text and the corresponding classroom courseware content text; Calculate the correlation between the classroom audio text and the classroom courseware content text; If the correlation is less than or equal to a first preset threshold, the classroom audio text is determined as the classroom content text; If the correlation is greater than or equal to the second preset threshold, the classroom courseware content text is determined as the classroom content text, where the second preset threshold is greater than the first preset threshold. If the correlation is greater than the first preset threshold and less than the second preset threshold, the classroom audio text and the classroom courseware content text are determined as the classroom content text.

3. The method according to claim 1, characterized in that, The chapter search library is also used to store multiple associated text vectors, which are text vectors used to represent chapter-related text corresponding to chapter names, and the text vectors represent text in the form of feature vectors; The step involves performing similarity matching between the classroom content text and the chapter-related text corresponding to each chapter name in the chapter search database, and determining N candidate chapter names, including: Calculate the similarity between each of the plurality of associated text vectors and the text vector of the classroom content text to obtain a plurality of first similarities. The plurality of first similarities are the first similarities corresponding to the plurality of chapter names. The first similarities are used to reflect the text similarity between the chapter-related text corresponding to the chapter name and the classroom content text. The N chapter names with the highest similarity are selected as N candidate chapter names.

4. The method according to claim 3, characterized in that, The associated text vector includes dense vectors and sparse vectors. The dense vector is a vector extracted based on the text embedding model to represent the text as a whole. The sparse vector is a vector extracted based on the text embedding model to represent the importance of each word in the text. The sparse vector includes the weight value corresponding to each word. The calculation of the similarity between each of the plurality of associated text vectors and the text vector of the classroom content text yields a plurality of first similarities, including: Calculate the similarity between the target dense vector and the dense vector of the classroom content text to obtain the dense similarity corresponding to the target chapter name. The target dense vector is the dense vector of the chapter-related text corresponding to the target chapter name, and the target chapter name is any one of the multiple chapter names. Calculate the similarity between the target sparse vector and the sparse vector of the classroom content text to obtain the sparse similarity corresponding to the target chapter name. The target sparse vector is the sparse vector of the chapter-related text corresponding to the target chapter name. Based on the dense similarity and sparse similarity of each chapter name among the plurality of chapter names, the first similarity of each chapter name is determined, and the plurality of first similarities are obtained.

5. The method according to claim 4, characterized in that, The calculation of the similarity between the target dense vector and the dense vector of the classroom content text, to obtain the dense similarity corresponding to the target chapter name, includes: Calculate the dot product between the target dense vector and the dense vector of the classroom content text to obtain the dense similarity corresponding to the target chapter name.

6. The method according to claim 4, characterized in that, The calculation of the similarity between the target sparse vector and the sparse vector of the classroom content text, to obtain the sparse similarity corresponding to the target chapter name, includes: Determine the intersection between the first word set and the word set of the classroom content text to obtain the first intersection. The first word set is the word set of the chapter-related text corresponding to the target chapter name. The word set is a set composed of each word in the text. The first intersection includes at least one word. Calculate the product of the first weight value and the second weight value to obtain the sparse weight value corresponding to the first word element. The first weight value is the weight value of the first word element in the target sparse vector, and the second weight value is the weight value of the first word element in the sparse vector of the classroom content text. The first word element is any word element in the first intersection. The sparse weight values ​​corresponding to each word element in the first intersection are summed to obtain the sparse similarity corresponding to the target chapter name.

7. The method according to claim 1, characterized in that, The chapter search library is also used to store multiple name text vectors, which are text vectors used to represent chapter names, and the text vectors represent text in the form of feature vectors; The step of matching the N candidate chapter names with the classroom content text based on similarity and determining the classroom chapter names includes: Calculate the similarity between the text vector of the target name and the text vector of the classroom content text to obtain the second similarity corresponding to the target candidate chapter name. The text vector of the target name is the text vector of the target candidate chapter name, and the target candidate chapter name is any one of the N candidate chapter names. The second similarity is used to reflect the text similarity between the chapter name and the classroom content text. Based on the second similarity of each of the N candidate chapter names, the candidate chapter name with the highest second similarity is determined as the classroom chapter name.

8. The method according to claim 7, characterized in that, The name text vector includes a sparse vector, which is a vector extracted based on a text embedding model to represent the importance of each word in the text, and the sparse vector includes the weight value corresponding to each word. The calculation of the similarity between the text vector of the target name and the text vector of the classroom content text, to obtain the second similarity corresponding to the target candidate chapter name, includes: Determine the intersection between the second word set and the word set of the classroom content text to obtain the second intersection. The second word set is the word set of the target candidate chapter name. The word set is a set composed of each word in the text. The second intersection includes at least one word. Calculate the product of the third weight value and the fourth weight value to obtain the sparse weight value corresponding to the second word element. The third weight value is the weight value of the second word element in the target name text vector, and the fourth weight value is the weight value of the second word element in the sparse vector of the classroom content text. The second word element is any word element in the second intersection. The sparse weight values ​​corresponding to each word in the second intersection are summed to obtain the second similarity corresponding to the target candidate chapter name.

9. The method according to claim 4 or 8, characterized in that, The text embedding model is obtained by fine-tuning a preset embedding model using positive and negative sample data. The preset embedding model is used to convert text into text vectors. The positive sample data includes preset difficult samples and the first standard courseware text corresponding to the preset difficult samples. The negative sample data includes the preset difficult samples and the second standard courseware text corresponding to the preset difficult samples. The preset difficult samples are audio text or courseware content text that the preset embedding model cannot process. The first standard courseware text is the content text of the first standard courseware, and the chapter name corresponding to the first standard courseware is the same as the actual chapter name corresponding to the preset difficult samples. The second standard courseware text is the content text of the second standard courseware, and the chapter name corresponding to the second standard courseware is different from the actual chapter name corresponding to the preset difficult samples.

10. The method according to any one of claims 1-8, characterized in that, The associated text of the chapter includes at least two of the following: courseware content text, courseware description text, and courseware content keywords. The courseware description text is text that expresses the courseware content in colloquial language.

11. The method according to claim 10, characterized in that, 所述 The courseware description text corresponding to each chapter name in the chapter search database is obtained in the following way: Get screenshots of the standard courseware corresponding to the chapter titles; The courseware screenshot prompt information and model prompt text are input into the preset multimodal graphic-text large model for text generation, resulting in courseware description text corresponding to the chapter name. The courseware screenshot prompt information includes the standard courseware screenshot and the screenshot prompt text corresponding to the standard courseware screenshot. The model prompt text is the prompt text that prompts the multimodal graphic-text large model to generate conversational text.

12. The method according to claim 10, characterized in that, The courseware content keywords corresponding to each chapter name in the chapter search database are obtained through the following method: Hot keywords are extracted from the content of all courseware corresponding to the chapter title to obtain the first set of keywords corresponding to the chapter title. The hot keywords are representative keywords in the courseware content. Keyword extraction is performed on the content of all popular courseware corresponding to the chapter title to obtain a second set of keywords corresponding to the chapter title. The popular courseware refers to representative courseware. The keywords in the first keyword set and the keywords in the second keyword set are determined as the keywords corresponding to the chapter names.

13. A classroom chapter recognition device, characterized in that, include: The text acquisition module is used to acquire classroom content text, which includes classroom audio text and / or classroom courseware content text corresponding to the classroom audio text. The classroom audio text is obtained by automatic speech recognition of the classroom audio. The first matching module is used to perform similarity matching between the classroom content text and the chapter-related text corresponding to each chapter name in the chapter search library, and to determine N candidate chapter names. The chapter search library is used to store multiple chapter names, and the chapter-related text is text associated with the chapter name to introduce the chapter knowledge. The chapter-related text corresponding to the N candidate chapter names has the highest matching degree with the classroom content text, and N is a positive integer greater than or equal to 2. The second matching module is used to perform similarity matching between the N candidate chapter names and the classroom content text, and to determine the classroom chapter name, which is the candidate chapter name with the highest matching degree with the classroom content text.

14. A computer device, characterized in that, The device includes a memory and a processor, the memory being connected to the processor, the processor being configured to execute one or more computer programs stored in the memory, the processor causing the computer device to perform the method as described in any one of claims 1-12 when executing the one or more computer programs.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1-12.

Images