Professional knowledge base information labeling method and system, electronic equipment and storage medium

Abstract

The invention relates to an information labeling method and system for a professional knowledge base, electronic equipment and a storage medium. The method comprises the following steps: acquiring an original file for labeling, wherein the original file comprises a document file, an image file and / or an audio / video file; preprocessing the original file to obtain target data; segmenting the target data to obtain a plurality of knowledge units; and performing multi-dimensional labeling on each knowledge unit to obtain corresponding labeling information, the dimensions including an information source, a subject classification, a knowledge type and a keyword. According to the method, the content of the professional knowledge base is subjected to information labeling, so that the knowledge base which is high in quality, structured and easy to retrieve is constructed, and an accurate retrieval result with source basis can be provided for a user.

Description

Information annotation methods, systems, electronic devices, and storage media for professional knowledge bases Technical Field

[0001] This invention relates to the field of knowledge base construction technology, and in particular to a method, system, electronic device and storage medium for information annotation of a professional knowledge base. Background Technology

[0002] With the widespread use of the internet, acquiring professional knowledge has become increasingly easier. For example, people can use search engines to find various professional knowledge online; with the rise of artificial intelligence and the increasing sophistication of large language models, people can also acquire professional knowledge through artificial intelligence. However, the information on the internet is vast and complex, and information found through search engines carries the risk of being outdated, incomplete, or even erroneous. People need to invest extra effort to verify the credibility of the information. Furthermore, the information found through search engines is scattered, redundant, and lacks systematic organization, requiring users to select and organize from a large number of search results, which is time-consuming and laborious. In addition, the quality of search results returned by search engines is closely related to the keywords used by the user and their filtering capabilities. When a user's search capabilities are limited, the search results often fail to meet their needs. While acquiring knowledge through artificial intelligence is simple and convenient, the content returned by artificial intelligence is usually processed by large language models, which can lead to a certain degree of illusion, and the returned results are closely related to the user's questions and suggestions. In addition to the above methods, people can also obtain professional materials through libraries, academic databases, or publications. The knowledge acquired in this way is systematic and highly credible, but it has a high barrier to entry, lacks timeliness, and ordinary people often find it difficult to quickly locate the content they need from professional literature.

[0003] Therefore, it is evident that providing a high-quality, easily searchable knowledge base is of great significance. Summary of the Invention

[0004] In view of the technical problems existing in the prior art, the present invention proposes an information annotation method, system, electronic device and storage medium for a professional knowledge base, which is used to provide high-quality and easy-to-retrieve professional knowledge base content.

[0005] To address the aforementioned technical problems, according to one aspect of the present invention, an information annotation method for a professional knowledge base is provided. The method includes: acquiring an original file for annotation, wherein the original file is of the type of document, image, audio, or video; preprocessing the original file to obtain target data; segmenting the target data to obtain multiple knowledge units; and performing multi-dimensional annotation on each knowledge unit to obtain corresponding annotation information, wherein the dimensions include information source, topic classification, knowledge type, and keywords.

[0006] Optionally, the original file of type document or image is an electronic file obtained by digitizing a physical paper book, the electronic file including multiple pages; or, the original file of type document or image is an electronic book, the electronic book including multiple pages; correspondingly, the step of preprocessing the original file to obtain the target data includes: preprocessing the pages and classifying the layout areas, segmenting the pages according to the area classification to obtain the page text area, header area, footer area, and left and right margin areas; performing text area detection on the page text area to obtain one or more text areas and / or one or more image areas; performing layout analysis on each text area to obtain the layout attributes of each text area; performing character recognition on each text area, and classifying the recognized characters according to the text area. The layout attributes of the domain are formatted to obtain the first text with the same layout as the text area; multiple first texts are merged according to their original text area positions to obtain the page body text of a page; edge detection or connected component detection is performed on the image area to determine the image boundary; the image area is cut based on the determined image boundary to obtain the image in the image area; character recognition is performed on the header area, footer area, and left and right margin areas to obtain the page attribute text; corresponding text tags and image tags are created for the page body text and image of each page, wherein the page attribute text is used as the tag content of the text tag and image tag, respectively; the page body text and / or image of each page and their respective tags are stored in the order of the book pages from front to back to obtain the target data.

[0007] Optionally, after obtaining the first text and page attribute text through character recognition, the method further includes: performing error correction processing on the first text and page attribute text based on dictionary matching and semantic analysis.

[0008] Optionally, when multiple text regions and one or more image regions are obtained by performing text region detection on the main text area of ​​the page, the method further includes: detecting whether each text region conforms to the text annotation features of the image; determining the text region that conforms to the text annotation features of the image as the target text region and obtaining the corresponding first text; determining the target image region corresponding to the target text region based on the position of the target text region and the position of the image region; and adding the first text to the image tag content of the image in the target image region.

[0009] Optionally, the step of determining the corresponding target image region based on the position of the target text region and the position of the image region includes: when the position of an image region and the position of a target text region conform to the figure number labeling feature, the image region is determined to be the target image region corresponding to the target text region; when the overall position of a target text region combined with multiple consecutive image regions conforms to the figure number labeling feature, each of the multiple consecutive image regions is taken as the target image region corresponding to the target text region.

[0010] Optionally, after obtaining the image tag content, the method further includes: using the first text of the remaining text area, the page text of the previous page, or the page text of the next page as the search scope, and searching for image tag content within the search scope; in response to finding image tag content in a text area within the search scope, establishing an association link between the image and the text in the text area.

[0011] Optionally, when the original file is audio, the step of preprocessing the original file to obtain the target data includes: obtaining the information source of the original audio file and storing it as information source text, wherein the information source includes the professional field, topic name, speaker and time of speech; performing speech recognition on the original audio file to obtain second text, wherein the second text includes a text timestamp; and performing proofreading on the second text to obtain body text, wherein the proofreading includes removing interjections, repeated words and repeated text, wherein the body text includes the same text timestamp as the second text; wherein the information source text and the body text of the original audio file constitute the target data.

[0012] Optionally, when the original file is a video, the step of preprocessing the original file to obtain target data suitable for retrieval includes: obtaining the information source of the original video file and storing it as information source text, wherein the information source includes the professional field, topic name, speaker, and time of the speech; performing audio-video separation on the original video file to obtain an audio file and a video file; performing speech recognition on the separated audio file to obtain a second text, wherein the second text includes a text timestamp; performing proofreading on the second text to obtain a third text, wherein the proofreading includes removing interjections, repeated words, and repeated text, wherein the third text includes the same text timestamp as the second text; and performing preprocessing on the separated video file... The system performs frame extraction processing on the original video file and performs image analysis on the extracted video frames. When the image analysis indicates that the video frame contains the content of the display file, a first video segment of the display file is extracted from the original video file. The first video segment is then divided into multiple video frames. The display image and the start and end times of the corresponding display duration are extracted from the video frames. The corresponding text segments are determined from the third text based on the start and end times. A link is created between the display image and the text segments, wherein the third text, the display image, and the link between the display image and the text segments constitute the main text of the original video file. The information source text and the main text of the original video file constitute the target data.

[0013] Optionally, the step of segmenting the target data to obtain multiple knowledge units includes: inputting the target data and prompt text into a large language model; wherein the prompt text includes at least a knowledge unit definition, a segmentation granularity description, and an output format of the knowledge unit; the large language model, based on the knowledge unit definition and segmentation granularity description in the prompt text, segments the target data in a forward-to-back order to obtain multiple knowledge unit texts and their position information in the original file, and outputs the knowledge unit texts and their position information according to the output format in the prompt text.

[0014] Optionally, the output format of the knowledge units in the prompt text is a table. Correspondingly, the prompt text defines the table header, which includes the knowledge unit text identifier, the knowledge unit text, the position of the knowledge unit in the original file, and the label dimension name. Correspondingly, the large language model creates a table according to the table header defined in the prompt text and names the table title with the name of the original file. When segmenting knowledge units, the large language model fills the identifier, knowledge unit text, and position information of each knowledge unit into the corresponding cell of the table header according to the segmentation order, thereby obtaining the knowledge unit table of the original file.

[0015] Optionally, when performing multi-dimensional annotation on each knowledge unit, the steps for annotating the information source include: obtaining the page text and text tag content of a first page of a first number from front to back in the target data, and / or obtaining the page text and text tag content of a second page of a second number from back to front; extracting the field content of a first preset field from the page text and text tag content of the first page and / or the page text and text tag content of the second page; wherein the first preset field includes author, book title, and publication information; generating structured first information based on multiple first preset fields and their field content; extracting the field content of a second preset field from the location information of each knowledge unit; wherein the second preset field includes at least page number and line number; generating structured second information based on the second preset field and its field content; and merging the first information with the second information of each knowledge unit to constitute the information source content of the knowledge unit.

[0016] Optionally, when performing multi-dimensional annotation on each knowledge unit, the steps for annotating the information source include: obtaining the information source text from the target data; extracting the field content of a first preset field from the information source text; wherein the first preset field includes at least the speaker, topic name, and occurrence time; generating structured first information based on multiple first preset fields and their field content; extracting the content of a second preset field from the location information of each knowledge unit, wherein the second preset field includes at least the duration and start time; generating structured second information based on the second preset field and its field content; and merging the first information with the second information of each knowledge unit to constitute the information source content of the knowledge unit.

[0017] Optionally, when performing multi-dimensional annotation on each knowledge unit, the steps for annotation topic classification include: obtaining a knowledge system tree of the professional field to which the original file belongs, wherein the knowledge system tree includes multiple nodes at level one and above from the root node to the leaf node, and the node label of each node includes a node classification topic and a classification feature description; obtaining a first preset number of page texts from the target data from front to back, classifying the page texts of the first pages according to the node classification topics in the knowledge system tree to obtain a first classification topic, wherein the first classification topic includes multiple node classification topics arranged in order from superior to subordinate nodes; obtaining the text of each knowledge unit, classifying the text of each knowledge unit according to the node classification topics in the knowledge system tree to obtain a second classification topic, wherein the second classification topic includes one or more node classification topics arranged in order from superior to subordinate nodes; and concatenating the first classification topic and the second classification topic together according to the node level order in the knowledge system tree to form the topic classification of the knowledge unit.

[0018] Optionally, when performing multi-dimensional annotation on each knowledge unit, the steps for annotation topic classification include: obtaining the professional field and topic name from the information source text; obtaining the knowledge system tree of the professional field, wherein the knowledge system tree includes multiple nodes at level one and above from the root node to the leaf node, and the node label of each node includes the node classification topic and the topic classification feature description; semantically matching the topic name with the node classification topic of the knowledge system tree to obtain the first node; obtaining multiple second nodes from the first node to the root node based on the hierarchical relationship of the nodes in the knowledge system tree; arranging the node classification topics of the second nodes and the node classification topics of the first nodes in the order of nodes from superior to subordinate to obtain the first classification topic; obtaining the text of each knowledge unit, classifying each knowledge unit text according to the node classification topic in the knowledge system tree to obtain the second classification topic, wherein the second classification topic includes one or more node classification topics arranged in the order of nodes from superior to subordinate; and concatenating the first classification topic and the second classification topic together in the order of node levels in the knowledge system tree to form the topic classification of the knowledge unit.

[0019] Optionally, when performing multi-dimensional annotation on each knowledge unit, the steps for annotating the knowledge type include: obtaining the text of each knowledge unit, and classifying the text of each knowledge unit according to a pre-defined knowledge type classification system to obtain the specific knowledge type.

[0020] Optionally, when annotating each knowledge unit in multiple dimensions, the steps for annotating keywords include: obtaining the text of each knowledge unit and extracting keywords from the knowledge unit text based on a keyword extraction model.

[0021] According to another aspect of the present invention, the present invention also provides an information annotation system for a professional knowledge base, the system comprising: a data acquisition module configured to acquire original files for annotation, the original files being of the type of document, image, audio, or video; a preprocessing module configured to preprocess the original files to obtain target data; a segmentation module configured to segment the target data to obtain multiple knowledge units; and an annotation module configured to perform multi-dimensional annotation on each knowledge unit to obtain corresponding annotation information, wherein the dimensions include information source, topic classification, knowledge type, and keywords.

[0022] According to another aspect of the present invention, an electronic device is also provided, including a processor and a memory, wherein a set of computer program instructions is stored in the memory, and the aforementioned information annotation method for a professional knowledge base is implemented when the processor executes the set of computer program instructions in the memory.

[0023] According to another aspect of the present invention, the present invention also provides a computer-readable storage medium, wherein a computer program instruction set is stored on the computer-readable storage medium, and the computer program instruction set, when executed by a processor, implements the aforementioned information annotation method for a professional knowledge base.

[0024] According to another aspect of the present invention, the present invention also provides a computer program product comprising a computer program instruction set, which, when executed by a processor, implements the aforementioned information annotation method for a professional knowledge base.

[0025] The information annotation method and system provided by this invention can generate structured, high-quality, and easy-to-retrieve professional knowledge base content. When users use the professional knowledge base to retrieve information, it can provide users with accurate and source-based search results. Attached Figure Description

[0026] The preferred embodiments of the present invention will now be described in further detail with reference to the accompanying drawings, in which: Figure 1 is a flowchart of an information annotation method for a professional knowledge base according to an embodiment of the present invention; Figure 2 is a flowchart of a method for preprocessing original documents according to an embodiment of the present invention; Figure 3 is a schematic diagram of text descriptions located below an image according to an embodiment of the present invention; Figure 4 is a schematic diagram of text descriptions located above an image according to an embodiment of the present invention; Figure 5 is a schematic diagram of text descriptions located beside an image according to an embodiment of the present invention; Figure 6 is a flowchart of a method for recognizing text descriptions in an image according to an embodiment of the present invention; Figure 7 is a schematic diagram of a target text region and a target image region according to an embodiment of the present invention; Figure 8 is a schematic diagram of a target text region and a target image region according to another embodiment of the present invention; Figure 9 is a schematic diagram of a method for recognizing text descriptions in an image according to an embodiment of the present invention; Figure 9 is a schematic diagram of a method for recognizing text descriptions in an image according to an embodiment of the present invention; Figure 1 is a flowchart ... Figure 10 is a flowchart of a method for recognizing text descriptions of images according to another embodiment of the present invention; Figure 11 is a flowchart of a method for preprocessing original files according to yet another embodiment of the present invention; Figure 12 is a flowchart of a method for labeling the information source of knowledge units according to an embodiment of the present invention; Figure 13 is a flowchart of a method for labeling the topic classification of knowledge units according to an embodiment of the present invention; Figure 14 is a schematic diagram of a knowledge system tree in the field of education according to an embodiment of the present invention; Figure 15 is a flowchart of a method for labeling the topic classification of knowledge units according to another embodiment of the present invention; Figure 16 is a block diagram of the principle of an information labeling system for a professional knowledge base according to an embodiment of the present invention; Figure 17 is a schematic diagram of the hardware structure principle of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] In the following detailed description, reference can be made to the accompanying drawings, which form part of this application and illustrate specific embodiments of the present application. In the drawings, similar reference numerals describe substantially similar components in different figures. Specific embodiments of the present application are described in sufficient detail below to enable those skilled in the art to implement the technical solutions of the present application. It should be understood that other embodiments may also be utilized, or structural, logical, or electrical changes may be made to the embodiments of the present application.

[0029] This invention provides a method and system for annotating information in a professional knowledge base. By annotating the content of the professional knowledge base, a high-quality, structured, and easily searchable knowledge base is constructed, thereby providing users with accurate and source-based search results under the processing of search engines. Referring to Figure 1, Figure 1 is a flowchart of an information annotation method for a professional knowledge base according to an embodiment of the present invention, wherein the method includes the following steps: Step S1, obtaining the original file.

[0030] Step S2: Preprocess the original file to obtain the target data.

[0031] Step S3: The target data is segmented to obtain multiple knowledge units (Chunks).

[0032] Step S4: Perform multi-dimensional annotation on each knowledge unit to obtain corresponding annotation information, wherein the dimensions include information source, topic classification, knowledge type and keywords.

[0033] In step S1, the original files are various document or image types, such as scanned copies of e-books or printed books. Document files may be PDF documents, and image files may be JPG images. These e-books and printed books are core works and research papers in their respective professional fields. The original files in step S1 may also be audio / video files, such as recordings of academic conferences, conference videos, or screen recordings of online classes. These audio / video files record the speaker's remarks.

[0034] Furthermore, the original documents here are the specific content selected during the construction of the professional knowledge base. To construct a high-quality, comprehensive, and systematic professional knowledge base, this invention follows the principles of "core priority, authoritative reliability, multi-faceted complementarity, and copyright compliance" when determining the specific content of the knowledge base. First, the core knowledge system of the professional knowledge base is determined. This core knowledge system includes two aspects: one is core monographs within the professional field, namely, including recognized "bible"-level monographs and foundational works in the field; the other is the tracking content of authoritative authors, namely, systematically including representative works and all research results of top scholars in the field, including the latest research results. Then, the data sources for the professional knowledge base are determined. For example, digitized authoritative books and journals can be obtained from authoritative digital resource databases, such as university libraries, CNKI, Wanfang Data, JSTOR, Google Scholar, and other academic databases. Paper books can also be digitized; for example, for important monographs that have not yet been digitized, high-quality scanning and professional OCR (Optical Character Recognition) technology are used for text digitization, followed by rigorous manual proofreading to ensure the accuracy of the text. It can also obtain audio or video files of important conferences from academic conference organizations within the professional field. When acquiring content for the professional knowledge base, this invention prioritizes books in the public domain or authorized digital resources. When handling copyrighted materials, it strictly adheres to the principle of fair use and clearly records copyright information to ensure the legality of the database.

[0035] Referring to Figure 2, Figure 2 is a flowchart of a method for preprocessing an original document according to an embodiment of the present invention. In this embodiment, a paper book is scanned page by page to obtain an image file containing multiple pages, and the original document used as the annotation object is the image file. During the preprocessing of the original document, each page is processed from front to back to obtain the text, image, and corresponding tag content of each page. The text, images, and corresponding tag content of all pages constitute the target data for subsequent processing. Taking a page containing both text and images as an example, the processing method for a single page includes the following steps: Step S211, preprocessing and classifying the page layout. The preprocessing includes image binarization, noise reduction, and tilt correction. Since the annotation object in this embodiment is a scanned image file of a book, the present invention classifies the page layout based on the characteristics of the book.

[0036] Step S212: Divide the page according to the layout area to obtain the main text area, header area, footer area, and left and right margin areas.

[0037] Step S213 involves performing text region detection on the main text area of ​​the page to obtain one or more text regions and one or more image regions. Since the main text area of ​​the page is where the main content of the book is distributed, and this content may consist entirely of text, entirely of images, or a combination of both, the content type of the main text area can be determined through text region detection. In this embodiment, each line of text is defined as a text region during text region detection.

[0038] Step S214: Perform layout analysis on each text area to obtain the layout attributes of each text area, such as text area number (i.e., text line number), font size, paragraph indentation position, and other attribute information.

[0039] Step S215: Perform character recognition on each text region and arrange the recognized characters according to the layout attributes of the text region to obtain the first text with the same layout as the text region.

[0040] Step S216: Merge multiple first texts according to the original text region position of each first text to obtain the page body text of a page.

[0041] Step S217: Perform edge detection or connected component detection on the image region to determine the image boundary.

[0042] Step S218: The image region is segmented based on the determined image boundaries to obtain the image in the image region.

[0043] Step S219: Perform character recognition on the header area, footer area, and left and right margin areas respectively to obtain page attribute text. Page attribute text may include one or more of the following: the book chapter of the page in the header area, the page number in the footer area, and the annotation text in the left and right margin areas.

[0044] Step S2110: Create corresponding text tags and image tags for each page's main text and images, using page attribute text as tag content. In this embodiment, when a page contains both text and images, the page includes text tags and image tags. The tag content includes, for example, book chapters, page numbers, etc., thereby indicating the specific location of the main text and images within the book.

[0045] After obtaining the main text, images, and corresponding tags for each page, the main text, images, and their respective tags for each page are stored in the order of the book pages from front to back to obtain the target data.

[0046] In practical applications, when there are no images on the page, only the main text of the page and its tags are obtained; when there are only images on the page, only the images and their tags are obtained.

[0047] In Figure 2, the target data is obtained in units of pages, including the main text, images, and corresponding tags of a page. In another embodiment, the target data can also be in units of paragraphs. Therefore, in step S214, when performing layout analysis on each text area, the layout attributes of each text area include text area position, font size, line number, paragraph indentation, and end position, etc. Then, in step S216, according to the paragraph indentation and end position of each first text, two or more adjacent first texts that can form a paragraph are merged to obtain a page main text block, i.e., a paragraph, and the line number and line number of the text block are recorded as text block attribute information. Then, in step S2110, the page attribute text and text block attribute information are used together as tag content. In this embodiment, when a page contains both text and images, the page includes text tags and image tags. The tags include book chapters and page numbers, thereby marking the specific location of the main text block and images in the book.

[0048] After obtaining all the main text blocks, images, and corresponding tags for each page, store the main text blocks, images, and their respective tags for each page in the order of the book pages from front to back to obtain the target data.

[0049] In this embodiment, the text in the book is stored in units of paragraphs, and the paragraph text is marked with chapter, page number, and line number to facilitate retrieval.

[0050] Furthermore, in the aforementioned step S215, after obtaining the first text through character recognition, error correction processing is performed on the first text based on dictionary matching and semantic analysis, thereby improving the accuracy of recognition. Similarly, after obtaining the page attribute text, error correction processing is performed on the page attribute text based on dictionary matching and semantic analysis.

[0051] Images in books are usually accompanied by brief text descriptions. These descriptions are typically located below or above the images, as shown in Figures 3 and 4. Figure 3 is a schematic diagram of text descriptions located below an image according to an embodiment of the present invention, and Figure 4 is a schematic diagram of text descriptions located above an image according to an embodiment of the present invention. These descriptions may be figure number markings, such as "Figure 1," "Figure 1-2," or figure number markings and descriptions, such as "Figure 1 Transistor Input Characteristic Curve." The text descriptions may also be located next to the images, indicated by indicative icons to show the corresponding images, as shown in Figure 5, which is a schematic diagram of text descriptions located next to images according to an embodiment of the present invention. Therefore, these image descriptions should be added to the image labels, rather than being included as body text along with other main text content. Therefore, when multiple text regions and one or more image regions are obtained by text region detection in step S213 of the page's main text area, a step of recognizing the image text descriptions is also included, as shown in Figure 6, which is a flowchart of a method for recognizing image text descriptions according to an embodiment of the present invention. The method includes: step S220, determining a text region as the processing object.

[0052] Step S221: Detect whether the processed object conforms to the text annotation features of an image. The text annotation features are, for example, features used to annotate figure numbers such as "Figure 1-1" and "Figure 12," including text region length, indentation at the starting position, and distance between the ending position and the edge of the page area. If the text region length is less than a threshold (e.g., 3-10 character distances), the indentation at the starting position is greater than a threshold (e.g., two character distances), and the distance between the ending position and the edge of the area is greater than a threshold (e.g., 5-20 character distances), then the text in the current text region is considered image annotation text, and step S222 is executed. If the processed object does not conform to the image text annotation features, the detection of the current text region ends, and step S235 is executed.

[0053] Step S222: Identify the text region that matches the text annotation features of the image as the target text region and obtain the corresponding first text. For example, the text in this text region can be identified using OCR recognition.

[0054] Step S223: Determine the corresponding target image region based on the position of the target text region and the position of the image region. For example, compare the position of the target text region with the position of the image region to determine the image region adjacent to the target text region, and determine whether the positional relationship between the target text region and the image region conforms to the figure number annotation feature. The figure number annotation feature is, for example, that the target text region is located directly below or above the image region. If it conforms to the figure number annotation feature, the current image region can be determined as the target image region. The figure number annotation feature also includes an indicative icon, such as the triangle symbol in Figure 5. When the target text region includes an indicative icon before or after it, and the direction indicated by the icon identifies the image region, then the image region is designated as the target image region.

[0055] Step S224: Add the first text to the image tag content of the image in the target image area.

[0056] Step S225: Determine if there are any text regions that have not been detected. If so, return to step S230. If all text regions on the current page have been detected, end the processing.

[0057] In Figure 6, further, when a target text region is determined in step S222 and multiple consecutive image regions that conform to the image number labeling characteristics are determined in step S223, the multiple consecutive image regions are merged into a single image region, so that an image is obtained when extracting the image in step S218. Then, the first text identified from the target text region is added to the image label content of the image. As shown in Figure 7, Figure 7 is a schematic diagram of the target text region and the target image region according to an embodiment of the present invention. At this time, image regions 20, 21, and 22 are merged into a single image region, and an image is extracted. The text identified in the target text region 13 is added to the image label content of the image.

[0058] In another scenario, after determining multiple target text regions in step S222 and identifying a corresponding target image region for each target text region in step S223, if there is one remaining target text region, the multiple horizontally parallel and consecutive image regions are treated as a single image region. The position of the remaining target text region is then compared with the position of the single image region to determine if they conform to the image number labeling features. If they do, the first text identified in the remaining target text region is added to the image label content of each image. As shown in Figure 8, Figure 8 is a schematic diagram of target text regions and target image regions according to another embodiment of the present invention. In this embodiment, target image regions 20, 21, and 22 correspond to target text regions 10, 11, and 12, respectively. One image is extracted from each target image region and combined with its corresponding target text regions 10, 11, and 12, and 13, respectively, and added to the image label content.

[0059] Referring to Figure 9, which is a flowchart of a method for recognizing text descriptions of images according to another embodiment of the present invention. In this embodiment, starting from the first image region of the current page, the corresponding text description is determined from the text region for each image region. The method includes: step S230, taking the first image region of the current page as the target image region.

[0060] Step S231: Determine the text regions adjacent to the target image region based on its location.

[0061] Step S232: Take an adjacent text region as the processing object.

[0062] Step S233: Determine whether the object being processed matches the text annotation features of the image. If the object matches the text annotation features, proceed to step S234. If the object does not match the text annotation features, proceed to step S236.

[0063] Step S234: Determine the text region that matches the text annotation features of the image as the target text region and obtain the corresponding first text.

[0064] Step S235: Add the first text to the image tag content of the image in the target image area.

[0065] Step S236: Determine if there are any adjacent text regions that have not been processed. If so, return to step S232. If all adjacent text regions have been processed, it means that the image in the current image region does not have a corresponding text description, and the processing ends.

[0066] In the aforementioned embodiments, after obtaining the image tags, the following processing steps are further included: using the page text in the first text corresponding to the remaining text area, or the previous page, or the next page as the search range, searching for image tag content in the search range; when image tag content is found in a text area within the search range, establishing an association link between the image and the text in the text area.

[0067] By following the steps above, a link is established between the page text and the images, so that when the text related to the image is found on the page, the corresponding image can also be retrieved.

[0068] When collecting and constructing the knowledge base, this invention is not limited to physical books and digital books, but can also include audio and video materials, such as recordings and videos of important professional lectures, online course recordings and screen recordings, etc. These audio / video files typically include a description of the source of the information, usually in text format. For example, for materials on an online lecture, this includes the lecture's audio / video files and an introductory document containing information such as the lecture's title, speaker's name, field of expertise, and lecture date.

[0069] Referring to Figure 10, which is a flowchart of a method for preprocessing an original file according to another embodiment of the present invention, the original file in this embodiment is an audio file. The method for preprocessing the original file to obtain target data includes the following steps: Step S240, obtaining the information source of the audio file and storing it as information source text. For example, extracting content related to the information source from an introductory document, such as the professional field, topic name, speaker, and speaking time.

[0070] Step S241: Perform speech recognition on the audio file to obtain the second text and a text timestamp. During speech recognition, a speech recognition model based on a neural network or a large language model can be used. Simultaneously with obtaining the second text, its timestamp is also obtained; for example, its time information is marked by word or vocabulary.

[0071] Step S242 involves proofreading the second text to obtain the main text. This proofreading includes removing modal particles, repeated words, and duplicate text. During proofreading, the timestamp is not changed; that is, the timestamp of the main text is the same as the timestamp of the second text.

[0072] Step S243: Combine the source text and body text of the audio file to form the target data.

[0073] Referring to Figure 11, which is a flowchart of a method for preprocessing an original file according to another embodiment of the present invention, the original file in this embodiment is a video file. The method for preprocessing the original file to obtain target data includes the following steps: Step S250, obtaining the information source of the video file and storing it as information source text. For example, reading the video description document in the video resource and extracting content related to the information source, such as professional field, topic name, speaker, and presentation time.

[0074] Step S251: Perform audio-visual separation on the video file to obtain an audio file and a video file.

[0075] Step S2521: Perform speech recognition on the audio file to obtain the second text, wherein the second text includes a corresponding text timestamp.

[0076] Step S2522: Proofread the second text to obtain the third text. The proofreading process includes removing modal particles, repeated words, and repeated text. The third text includes the same text timestamp as the second text.

[0077] Step S2531: Perform frame extraction on the video file and perform image analysis on the extracted video frames.

[0078] Step S2532: When the video frame is found to include the content displayed in the PPT file based on image analysis, the first video segment of the PPT file is extracted from the original video file.

[0079] Step S2533: Perform frame segmentation on the first video segment to obtain multiple video frames.

[0080] Step S2534: Extract the PPT image and the start and end times of the corresponding presentation duration from the video frame.

[0081] Step S254: Determine the text segment corresponding to each PPT image from the third text based on the start time and end time.

[0082] Step S255: Create links between PPT images and text snippets, where the third text, PPT images, and links between PPT images and text snippets constitute the main text of the video file.

[0083] Step S256: Combine the source text and body text of the video file to form the target data.

[0084] In this embodiment, the video file is converted into text and images. Based on the correspondence between audio timestamps and image playback times, a link is established between the text and the images. When a user searches, if a certain piece of text is found, the search results returned to the user include both the text and the PPT image.

[0085] Although this embodiment uses the recognition of PPT images in a video as an example for illustration, it can be understood that when other types of files are displayed in the video, the aforementioned processing method can still be used, such as Word documents, PDF documents, video presentations, model presentations, etc.

[0086] In another embodiment, in the aforementioned video file processing flow, the video file is also stored in the knowledge base as metadata along with the processed knowledge unit text. Since the knowledge unit text is segmented from the target data and retains the corresponding text timestamp, when a user performs a search, the corresponding video segment can be extracted from the metadata and returned to the user based on the retrieved text and its timestamp. Therefore, according to the tagging method provided by this invention, the search results returned to the user include text, PPT images, and corresponding video segments.

[0087] Similarly, when the original file is an audio file, the audio file is also stored in the knowledge base. When a user searches the knowledge base, the search results returned to the user include text and the corresponding audio clips.

[0088] In step S3, when segmenting the target data to obtain multiple knowledge units, in one embodiment, the powerful reasoning and semantic analysis capabilities of a large language model are utilized to segment the target data into multiple knowledge units. Specifically, the large language model is first trained to segment the knowledge units.

[0089] For example, first, create a prompt text, which should include at least a task description, a definition of a knowledge unit, a description of the segmentation granularity, and the output format of the knowledge unit. The task description specifies that the task of the large language model is to segment the input data into knowledge units and output the segmented knowledge units according to the required format. To enable the large language model to segment reasonable knowledge units, the prompt text provides a corresponding definition of a knowledge unit, for example, defining a complete "knowledge point" or "thought unit" as a knowledge unit. Furthermore, based on the characteristics of the professional field, the number of tokens for a knowledge unit is also limited. For example, in the field of education, the number of tokens for a knowledge unit is 300-500. To facilitate subsequent retrieval, the task description requires the extraction of the knowledge unit's location in the original file. For example, when segmenting a book scanned at the page level, when segmenting knowledge units, the corresponding content, such as chapter, page number, and line number, should be extracted from text tags or image tags based on the knowledge unit's location. For audio / video files, when segmenting knowledge units, the corresponding text timestamps are retained, and the text timestamps serve as the location information for the knowledge units.

[0090] The content of the segmentation granularity description should be adapted to the professional field. For example, in the field of education, the segmentation granularity description may include the following: "a complete definition," "the core argument or a sub-argument of a theory," "a complete case analysis," "a brief introduction and conclusion of an experiment or study," and so on. In addition, to improve the segmentation accuracy of the large language model, some supplementary prompts can be added. For example, prompts can be given based on the characteristics of the granularity, such as adding a prompt "What is constructivism?" after "a complete definition," or prompts can be given based on the writing style, such as "a paragraph or several consecutive paragraphs," to avoid content being interrupted in the middle of sentences or paragraphs.

[0091] Regarding the output format of knowledge units, since the knowledge units in this invention are content of a professional knowledge base, the output format of the knowledge units can be determined according to the retrieval needs of the professional knowledge base's retrieval engine to facilitate retrieval. In one embodiment, the output format of the knowledge unit is a table, and the table headers in the prompt text are defined as the knowledge unit text identifier, the knowledge unit text content, the location information, and the annotation dimension name.

[0092] Then, a sample set is prepared, in which each sample includes the original data and the segmented knowledge units.

[0093] The sample set of prompt texts is input into the large language model, enabling the large language model to learn how to segment knowledge units based on the sample prompt texts.

[0094] After learning from a certain number of samples, the large language model is tested to see if it can accurately segment knowledge units.

[0095] During training, the prompt text can be modified to help the large language model better understand the task.

[0096] Once the accuracy of the large language model in segmenting knowledge units meets the requirements, it can be put into use.

[0097] In step S3, the prompt text and target data are input into the large language model. Based on the definition and segmentation granularity of knowledge units in the prompt text, the large language model segments the target data in a forward-to-back order, extracts position information, and outputs the knowledge unit text and its position information according to the output format in the prompt text. As mentioned earlier, when the output format, such as a table and table header, is determined in the prompt text, the large language model creates a table according to the table header defined in the prompt text and names the table title with the name of the original file. It creates an identifier for each knowledge unit according to the segmentation order and adds the knowledge unit identifier, knowledge unit text, and position information to the cells in the table, thus obtaining the knowledge unit table of the original file. Here, the position information corresponds to the original files of text and image types, which are extracted from text and image tags, such as chapters, page numbers, and line numbers. For audio and video original files, the position information is time information determined by timestamps, such as the xth minute or xth second in the audio / video.

[0098] To facilitate the retrieval of professional knowledge bases and improve retrieval efficiency, this invention, after dividing the content of the professional knowledge base into multiple knowledge units, also performs multi-dimensional annotation on the knowledge units. In one embodiment, the annotation dimensions include "information source", "topic classification", "knowledge type" and "keywords". Therefore, in the aforementioned embodiment, the header of the knowledge unit table output by the large model includes these dimension names respectively.

[0099] The "Source of Information" dimension includes several structured fields, which vary slightly depending on the type of the original document. For example, when the original document is a scanned copy of a paper book, the "Source of Information" fields include author, title, publication year, number of publications, chapter, and page range. When the original document is an audio or video recording of an academic conference proceedings, seminar, or online class, the "Source of Information" fields include speaker, topic, presentation / speech time, duration, and start and end times. Below is the "Source of Information" dimension label content for a knowledge unit in a book: "Author": [Jean Piaget]; "Title": [Children's Language and Thinking]; "Publication Year":

[1923] ; "Chapter": [Chapter 1: The Functions of Children's Language]; "Page Range": [xx-xx]. The field name is before the colon, and the field content is after the colon.

[0100] When the original document consists of multiple pages obtained by scanning a paper book page by page, the knowledge unit is obtained by segmenting the main text of the page after scanning the paper book. For fields such as author, title, publication year, and publication number, all knowledge units have the same content. For chapter and page number ranges, each knowledge unit has different content. Refer to Figure 12, which is a flowchart of a method for annotating the source of knowledge unit information according to an embodiment of the present invention. In this embodiment, when annotating the source of knowledge unit information, the knowledge units are annotated one by one in ascending order of their identifiers. Specifically, the following steps are included: Step S400, obtaining the main text of the first page of a first preset number from the target data from front to back. Since physical paper books usually have a copyright page, which indicates the author and publication information, and the copyright page is usually in the first few pages or the last few pages.

[0101] Step S401: Extract the field content of preset fields from the main text of the first page. In this embodiment, the preset fields include author, book title, publication year, and number of publications.

[0102] Step S402: Determine whether the content of the preset field has been extracted. If the content of the preset field has been extracted, proceed to step S405. If the content of the preset field has not been extracted or the required field content is missing, proceed to step S403.

[0103] Step S403: Obtain the page text of the second page of the second preset number from backward to forward.

[0104] Step S404: Extract the content of preset fields from the main text of the second page. For example, if only publication-related fields were extracted in step S401, then the author field content will be extracted in step S404.

[0105] Step S405: Generate structured first information based on preset fields and their content. For example, {Author: xx}. Here, curly braces distinguish different fields and their content, and a colon within the curly braces divides the information into two parts: the part before the colon is the field name, and the part after the colon is the field content.

[0106] After the aforementioned processing steps, the common field content of all knowledge units was obtained. For the field content unique to each knowledge unit, such as chapter and page number range, the following processing is performed: Step S406, obtain the location information of the knowledge unit.

[0107] Step S407: Extract the field content of another part of the field from the location information of the knowledge unit to obtain the second information.

[0108] Step S408: Combine the first information and the second information to form complete information source content.

[0109] Step S409: Generate information source dimension labels for each knowledge unit. The content of the information source dimension labels is the information source content obtained in step S408 above.

[0110] When knowledge unit text is stored in a table format, the information source dimension label content can also be stored in the table output by the large language model.

[0111] When the original file is an audio / video file, after the aforementioned processing of the audio / video file, the information source text of the audio / video file is obtained. Each knowledge unit also includes location information. Therefore, some field content can be extracted from the information source text, such as: speaker, topic name, publication time, total duration, etc. Another part of the field content can be extracted from the location information of the knowledge unit, such as start and end time, duration, speaker, etc. These field contents are combined together as the information source content of the knowledge unit. Then, information source tags are created as information source tag content, which can also be filled into the table output by the large language model.

[0112] The dimension "topic classification" includes multiple structured fields, which are associated with the professional field of the original document and the content of the current knowledge unit. See Figure 13, which is a flowchart of a method for labeling knowledge unit topic classification according to an embodiment of the present invention. When the original document in the embodiment is a scanned copy of a paper book, the method includes the following steps: Step S411, obtaining the knowledge system tree of the professional field to which the original document belongs. The knowledge system tree is a classification system obtained by integrating the knowledge system, logic, and framework of professional works from various dynasties, based on a certain classification standard. It includes multiple nodes at one level or higher from the root node to the leaf nodes, with a unique hierarchical relationship between nodes; the node label of each node includes the node classification topic and a description of the topic classification features. The professional works mentioned here can be books, articles, important lectures, etc. See Figure 14, which is a schematic diagram of the knowledge system tree of the educational professional field according to an embodiment of the present invention. In the field of education, the classification standard is based on the Ministry of Education's standards, with "Education" as the root node and its sub-disciplines as first-level nodes, such as "Principles of Education," "Curriculum and Teaching Theory," "Educational Psychology," "History of Education," "Sociology of Education," and "Comparative Education." The main research areas or intellectual trends in the development of each sub-discipline serve as second-level nodes. For example, under the first-level node "Educational Psychology," it can be divided into areas such as "Learning Theory," "Cognitive Development," "Motivation and Affect," and "Individual Differences," each of which can serve as a second-level node. Each second-level node can be further divided into multiple tertiary nodes. For instance, under the second-level node "Learning Theory," it can be divided into "Behaviorism," "Cognitivism," "Constructivism," and "Humanism." Tertiary nodes can be some of the aforementioned key theories and concepts, or they can be figures, models, etc. In constructing the knowledge system tree, this invention, while using existing classification standards to categorize works within the standard system, also incorporates works from different eras, thereby enriching the content of the knowledge base. To facilitate the determination of the classification themes of knowledge units, this invention creates node labels for each node in the knowledge system tree. The node label content includes the node classification theme and a description of the theme classification characteristics. For example, for a first-level node, its corresponding second-level discipline name is used as the node classification theme, and the code name in the classification standard is used as the theme classification characteristic description, or a brief introduction to the knowledge system of the second-level discipline is used as the theme classification characteristic description. For example, for the field of education, the node label of the root node has the node classification theme "Education," and its theme classification characteristic description is, for example, "Studying educational phenomena and problems, revealing educational laws." The node classification theme of the first-level node "Educational Psychology" is "Educational Psychology," and its theme classification characteristic description is, for example, "Studying the psychological laws in the educational process," or "880.27 - Studying the psychological laws in the educational process."Among them, 880.27 is the classification code in the subject classification and codes issued by the Ministry of Education of China, where 880 represents education and 27 represents educational psychology. Each key theory, concept, figure, and model under the second-level node is a child node, that is, a third-level node. Therefore, the topic name of the third-level node can be the corresponding theory name, concept name, figure name, model name, etc., and its corresponding topic classification feature description is, for example, a brief introduction of the specific content.

[0113] Once the knowledge system tree is determined, the node label content can be obtained. Typically, the first few pages of a book include an overview of the overall content, a preface, etc. For an uncategorized book, this content can be used to determine the subject matter to which the book belongs and the main branches or classifications within that subject.

[0114] Step S412: Obtain the page text of the first page from the target data in a first preset quantity from front to back.

[0115] Step S413: Classify the main text of the first page according to the node classification topics in the knowledge system tree to obtain a first classification topic. The first classification topic includes multiple node classification topics arranged in descending order of node level. Taking the field of education as an example, the first classification topic includes the topic name "Education" of the root node, the specific discipline name in the first-level node, such as "Educational Psychology", and the specific research direction, field or trend of thought in the second-level node, such as "Cognition and Development". Combine the node classification topics of these nodes in hierarchical order to form the first classification topic. For example, the first classification topic in the aforementioned embodiment is "Education - Educational Psychology - Cognition and Development".

[0116] Step S414: Obtain the text of each knowledge unit and classify each knowledge unit text according to the node classification topics in the knowledge system tree to obtain a second classification topic. The second classification topic includes one or more node classification topics arranged in order from superior to subordinate nodes. Since the knowledge unit text is a fragment from a book and is obtained according to the definition and granularity followed during segmentation, such as "a complete definition", "the core argument or a sub-argument of a theory", "a complete case analysis", "an introduction and conclusion of an experiment or research", etc., the knowledge unit text is semantically compared with the node classification topic name and topic classification feature description. The node or multiple nodes with the closest semantics are determined as the nodes or node combinations that are suitable for the knowledge unit text. The node classification topics of these nodes are combined together in order of level to form the second classification topic. Taking the field of education as an example, and using the three-level structure of the knowledge system tree as shown in Figure 14, after obtaining the topics of the two-level nodes in step S413, in step S414, it is only necessary to determine one of the multiple tertiary nodes under the second-level nodes. For example, in the second-level nodes, the node "Zone of Proximal Development (Vygotsky)" is determined in "Cognition and Development", so the second category topic is "Zone of Proximal Development (Vygotsky)".

[0117] Step S415: Combine the first and second category topics in a forward-to-back order to form the topic classification of the knowledge unit. For example, the topic classification of a knowledge unit is "Education - Educational Psychology - Cognition and Development - Zone of Proximal Development (Vygotsky)".

[0118] Step S416: Generate topic classification dimension labels for knowledge units. The topic classification of the knowledge unit is used as the content of the topic classification dimension labels, or the topic classification is filled into the table output by the large language model. For ease of retrieval, in one embodiment, structured data is used to represent topic classifications, and the corresponding topic level is represented according to the level of the node in the knowledge system tree. For example, the topic classification of the aforementioned knowledge unit is "Education - Educational Psychology - Cognition and Development - Zone of Proximal Development (Vygotsky)", which is represented by structured data as: "Level 1 Classification": [Educational Psychology]; "Level 2 Classification": [Cognition and Development]; "Level 3 Classification": [Zone of Proximal Development (Vygotsky)].

[0119] Referring to Figure 15, which is a flowchart of a method for labeling knowledge unit topic classification according to another embodiment of the present invention, the original file in this embodiment is an audio / video file, and the method includes the following steps: Step S421, obtaining the professional field and topic name of the original file from the information source text.

[0120] Step S422: Obtain the knowledge system tree of the professional field. For example, by matching the professional field with the label content of the root node of the knowledge system tree, the corresponding knowledge system tree is determined.

[0121] Step S423 involves semantically matching the topic name of the original file with the node label content of the knowledge system tree to obtain the first node. Since there are no specific standards for the topic name of a lecture, but it can still reflect the theme of the content, a node with the closest semantic meaning can be determined by semantically matching the topic name with the node label content of the knowledge system tree. For distinction, this is called the first node. For example, as shown in Figure 14, the first node could be a second-level node such as "Cognition and Development".

[0122] Step S424: Based on the hierarchical relationships between nodes in the knowledge system tree, obtain multiple second nodes from the first node to the root node. When the first node is "Cognition and Development", the second nodes are "Educational Psychology" and "Education".

[0123] Step S425: Arrange the node category topics of the second node and the node category topics of the first node in order from top to bottom to obtain the first category topic. For example, arranging the nodes "Cognition and Development", "Educational Psychology" and "Education" in order from top to bottom will result in the first category topic "Education - Educational Psychology - Cognition and Development".

[0124] Step S426: Obtain the text of each knowledge unit and classify each knowledge unit text according to the node classification topics in the knowledge system tree to obtain a second classification topic. The second classification topic includes one or more node classification topics arranged in descending order of node level. By semantically comparing the knowledge unit text with the node tag content of the third-level nodes, the node with the closest semantics is determined, for example, the third-level node "Information Processing Theory". Therefore, the second classification topic is "Information Processing Theory".

[0125] Step S427: Combine the first category topic and the second category topic in a forward-to-back order to form a topic classification of knowledge units. For example, the first category topic "Education - Educational Psychology - Cognition and Development" and the second category topic "Information Processing Theory" obtained above are combined to form the topic classification "Education - Educational Psychology - Cognition and Development - Information Processing Theory".

[0126] Step S428: Generate topic category dimension labels for each knowledge unit, and use the topic category of the knowledge unit as the content of the topic category dimension labels.

[0127] When labeling topic categories, the first category of topics is determined first, which is the common content of all knowledge units. Then, the second category of topics is obtained by semantic matching based on the unique content of the knowledge units.

[0128] This invention provides multiple knowledge types to indicate the content attributes of a knowledge unit. In one embodiment, each knowledge unit text is classified according to a pre-defined knowledge type classification system to obtain a specific knowledge type. The knowledge type classification system is adapted to a specific professional field; taking education as an example, the knowledge types and their meanings in the field of education are as follows: [Definition]: An explanation of a concept.

[0129] [Theory]: An explanation of a theoretical framework.

[0130] [Person]: An introduction to the life or thoughts of an educator.

[0131] [Case Study]: An analysis of a specific teaching or educational situation.

[0132] [Method]: An introduction to a teaching method, research method, or evaluation method.

[0133] [Controversial Viewpoint]: Different opinions within the academic community regarding a particular issue.

[0134] [Historical Event]: A significant event in the history of education development.

[0135] Based on the above knowledge types and their meanings, the knowledge unit text is evaluated to determine the closest one as the knowledge type of the knowledge unit, and knowledge type dimension labels are generated. The specific knowledge types obtained from the classification are used as the content of the knowledge type dimension labels.

[0136] In another embodiment, when segmenting knowledge unit text, the knowledge type classification system can be input into the large language model along with the knowledge unit text. The task description in the prompt text requires that, upon obtaining the knowledge unit text, the knowledge type be determined based on the provided knowledge type classification system and output along with the knowledge unit text. In yet another embodiment, when segmenting knowledge unit text, the storage location or URL of the knowledge type classification system, knowledge system tree, etc., can be input into the large language model. The task description in the prompt text specifies that the files in the storage location or URL are used as the reference standard for topic classification and knowledge type classification.

[0137] For the dimension "keywords," in one embodiment, a keyword extraction model can be used to extract keywords from the knowledge unit text and generate keyword dimension labels for each knowledge unit, using the extracted keywords as the keyword dimension label content. The keyword extraction model can be, for example, a machine learning / deep learning-based model, such as a sequence labeling model BiLSTM-CRF combining a Bidirectional Long Short-Term Memory Network (BiLSTM) and a Conditional Random Field (CRF), a supervised model consisting of a pre-trained model BERT and a Conditional Random Field, or a generative model, such as various large language models.

[0138] The following is an example of the segmentation and annotation of a knowledge cell, wherein the knowledge cell is a section of the second chapter, "Education and Communication," of the book "Democracy and Education."

[0139] Chunk ID: DE_001_001_002Chunk Text: "Society not only continues to exist through transmission and communication, but it can almost be said that society exists in transmission and communication. There is not only a literal connection between the words 'common,' 'community,' and 'communication,' people living in a community because they have something in common; and communication is the means by which they acquire that common thing. In order to form a community or society, what they must share are purposes, beliefs, expectations, knowledge—a common understanding—what sociologists call shared interests. These things cannot be passed from one person to another like bricks; nor can they be shared like a pie cut into pieces. Communication that ensures people's participation in a shared understanding fosters the same emotional and rational tendencies—the same methods of responding to expectations and demands." Multi-dimensional metadata annotation: Source information: "Author": [John Dewey]; Title: [Democracy and Education] Education); Publication year:

[1916] ; Chapter: [Chapter 1 Education is a necessity of life]; Section: [Section 2 Education and Communication]; Page range: [Pages 5-6].

[0140] Topic Taxonomy: First-level category: Principles of Education; Second-level category: Theory of the Essence of Education; Third-level category: Theory of the Social Function of Education.

[0141] Knowledge Type: [Theory] - This knowledge unit elaborates on Dewey's core theoretical views on the relationship between education and social communication. Keywords: [Social Communication], [Community], [Educational Transmission], [Social Survival], [Common Understanding], [Dewey], [Pragmatist Education].

[0142] According to another aspect of the present invention, the present invention also provides an information annotation system for a professional knowledge base. Referring to Figure 16, Figure 16 is a block diagram of the principle of an information annotation system for a professional knowledge base according to an embodiment of the present invention. The system includes: a data acquisition module 100, a preprocessing module 200, a segmentation module 300, and an annotation module 400. The data acquisition module 100 acquires original files, including document files, image files, and / or audio / video files. The preprocessing module 200 preprocesses the original files to obtain target data. As described in the foregoing method, when the original file is a scanned page of a book, the target data consists of the page text and / or images stored in the order of the book pages from front to back, along with their respective tags. The tag content is, for example, page attribute text. When the original file is an audio / video file, the information source text and the main text of the audio / video file constitute the target data. The segmentation module 300 segments the target data to obtain multiple knowledge units. A large language model can be used to segment the target data, and the knowledge unit text is output according to a set output format. The annotation module 400 performs multi-dimensional annotation on each knowledge unit to obtain corresponding annotation information. These dimensions include information source, topic classification, knowledge type, and keywords. For details, please refer to the aforementioned method section; further elaboration will not be repeated here.

[0143] Figure 17 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. The electronic device can be implemented as a server or other various terminal devices, such as desktop personal computers, tablet computers, laptop computers, mobile phones, etc. The electronic device includes a processor 601 and a memory 602. The memory 602 stores a program instruction set. When the processor 601 executes the program instruction set in the memory 602, the aforementioned information annotation method of the professional knowledge base is implemented.

[0144] Specifically, the processor 601 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0145] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. In a particular embodiment, memory 602 is a non-volatile solid-state memory.

[0146] The memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and other electrical, optical, or other physical / tangible memories. Typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the information annotation method for the professional knowledge base provided by this invention.

[0147] In one example, the electronic device may also include a communication interface 603 and a bus 604. The processor 601, memory 602, and communication interface 603 are connected via bus 604 and communicate with each other. Communication interface 603 is primarily used to implement communication between modules, devices, units, and / or equipment in the embodiments of the present invention. Bus 604 includes hardware, software, or both, coupling various components together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 604 may include one or more buses.

[0148] The present invention also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement any of the information annotation methods for a professional knowledge base in the foregoing embodiments. The computer-readable storage medium can be any tangible medium that contains or stores computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. The storage medium can be a transient computer-readable storage medium or a non-transitory computer-readable storage medium. Non-transitory computer-readable storage media may include, but are not limited to, magnetic storage devices, optical storage devices, and / or semiconductor storage devices. Examples of such storage devices include, for example, magnetic disks, optical discs based on CD, DVD, or Blu-ray technology, and persistent solid-state storage such as flash memory and solid-state drives.

[0149] This invention also provides a computer program product, comprising a set of computer program instructions, which, when executed by a processor, implement any of the information annotation methods for the professional knowledge base described in the foregoing embodiments. The computer program product includes, but is not limited to, application installation packages published on websites and in app stores, application plugins, and mini-programs that can run within certain applications.

[0150] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0151] The above embodiments are for illustrative purposes only and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the scope of the invention. Therefore, all equivalent technical solutions should also fall within the scope of the invention.

Claims

1. A method for annotating information in a professional knowledge base, characterized in that, include: Obtain the original file for annotation, wherein the original file type is document, image, audio or video; Preprocess the original file to obtain the target data; The target data is segmented to obtain multiple knowledge units; and each knowledge unit is annotated in multiple dimensions to obtain corresponding annotation information, wherein the dimensions include information source, topic classification, knowledge type and keywords.

2. The method according to claim 1, characterized in that, The original file, classified as a document or image, is an electronic file obtained by digitizing a physical paper book, and the electronic file includes multiple pages; or, the original file, classified as a document or image, is an electronic book, and the electronic book includes multiple pages. Correspondingly, the steps for preprocessing the original file to obtain the target data include: preprocessing the pages and classifying the layout areas; segmenting the pages according to the area classification to obtain the page text area, header area, footer area, and left and right margin areas; performing text area detection on the page text area to obtain one or more text areas and / or one or more image areas; performing layout analysis on each text area to obtain the layout attributes of each text area; and performing character recognition on each text area, and classifying the recognized characters according to the layout of the text area. The page attributes are formatted to obtain the first text with the same layout as the text area; multiple first texts are merged according to their original text area positions to obtain the page body text of a page; edge detection or connected component detection is performed on the image area to determine the image boundary; the image area is cut based on the determined image boundary to obtain the image in the image area; character recognition is performed on the header area, footer area, and left and right margin areas to obtain the page attribute text; corresponding text tags and image tags are created for the page body text and image of each page, wherein the page attribute text is used as the tag content of the text tag and image tag, respectively; and the page body text and / or image of each page and their respective tags are stored in the order of the book pages from front to back to obtain the target data.

3. The method according to claim 2, characterized in that, After obtaining the first text and page attribute text through character recognition, the process further includes: performing error correction processing on the first text and page attribute text based on dictionary matching and semantic analysis.

4. The method according to claim 2, characterized in that, When multiple text regions and one or more image regions are obtained by performing text region detection on the main text area of ​​the page, the method further includes: detecting whether each text region conforms to the text annotation features of the image; determining the text region that conforms to the text annotation features of the image as the target text region and obtaining the corresponding first text; determining the target image region corresponding to the target text region based on the position of the target text region and the position of the image region; and adding the first text to the image tag content of the image in the target image region.

5. The method according to claim 4, characterized in that, The step of determining the target image region corresponding to the target text region based on the position of the target text region and the position of the image region includes: when the position of an image region and the position of a target text region conform to the figure number labeling feature, the image region is determined to be the target image region corresponding to the target text region; and when the overall position of a target text region combined with multiple consecutive image regions conforms to the figure number labeling feature, each of the multiple consecutive image regions is taken as the target image region corresponding to the target text region.

6. The method according to claim 4 or 5, characterized in that, After obtaining the image tag content, the process further includes: using the first text of the remaining text area, the page text of the previous page, or the page text of the next page as the search scope, searching for image tag content within the search scope; and in response to finding image tag content in a text area within the search scope, establishing an association link between the image and the text in the text area.

7. The method according to claim 1, characterized in that, When the original file is audio, the steps for preprocessing the original file to obtain the target data include: obtaining the information source of the original audio file and storing it as information source text, wherein the information source includes the professional field, topic name, speaker and time of speech; performing speech recognition on the original audio file to obtain second text, wherein the second text includes a text timestamp; and performing proofreading on the second text to obtain body text, wherein the proofreading includes removing interjections, repeated words and repeated text, wherein the body text includes the same text timestamp as the second text; wherein the information source text and the body text of the original audio file constitute the target data.

8. The method according to claim 1, characterized in that, When the original file is a video, the steps for preprocessing the original file to obtain target data suitable for retrieval include: obtaining the information source of the original video file and storing it as information source text, wherein the information source includes the professional field, topic name, speaker, and time of the speech; performing audio-video separation on the original video file to obtain an audio file and a video file; performing speech recognition on the separated audio file to obtain a second text, wherein the second text includes a text timestamp; performing proofreading on the second text to obtain a third text, wherein the proofreading includes removing interjections, repeated words, and repeated text, wherein the third text includes the same text timestamp as the second text; and further processing the separated video file... The process involves frame extraction and image analysis of the extracted video frames. When the image analysis indicates that the video frame contains content from the display file, a first video segment containing the display file is extracted from the original video file. This first video segment is then divided into multiple video frames. The display image and the start and end times of the corresponding display duration are extracted from the video frames. Based on the start and end times, the corresponding text segments are determined from a third text source. A link is created between the display image and the text segments, where the third text, the display image, and the link between the display image and the text segments constitute the main text of the original video file. The source text and the main text of the original video file constitute the target data.

9. The method according to claim 2, 7, or 8, characterized in that, The step of segmenting the target data to obtain multiple knowledge units includes: inputting the target data and prompt text into a large language model; wherein the prompt text includes at least a knowledge unit definition, a segmentation granularity description, and an output format of the knowledge unit; and the large language model, based on the knowledge unit definition and segmentation granularity description in the prompt text, segmenting the target data in a forward-to-back order to obtain multiple knowledge unit texts and their position information in the original file, and outputting the knowledge unit texts and their position information according to the output format in the prompt text.

10. The method according to claim 9, characterized in that, The output format of the knowledge units in the prompt text is a table. Correspondingly, the prompt text defines the table header, which includes the knowledge unit text identifier, the knowledge unit text, the position of the knowledge unit in the original file, and the label dimension name. Correspondingly, the large language model creates a table according to the table header defined in the prompt text and names the table title with the name of the original file. When segmenting knowledge units, the large language model fills the identifier, knowledge unit text, and position information of each knowledge unit into the corresponding cell of the table header according to the segmentation order, thereby obtaining the knowledge unit table of the original file.

11. The method according to claim 9, characterized in that, When performing multi-dimensional annotation on each knowledge unit, the steps for annotating the information source include: obtaining the page text and text tag content of a first page of a first number from front to back in the target data, and / or obtaining the page text and text tag content of a second page of a second number from back to front; extracting the field content of a first preset field from the page text and text tag content of the first page and / or the page text and text tag content of the second page; wherein the first preset field includes author, title of work, and publication information; generating structured first information based on multiple first preset fields and their field content; extracting the field content of a second preset field from the location information of each knowledge unit; wherein the second preset field includes at least page number and line number; generating structured second information based on the second preset field and its field content; and merging the first information with the second information of each knowledge unit to form the information source content of the knowledge unit.

12. The method according to claim 9, characterized in that, When performing multi-dimensional annotation on each knowledge unit, the steps for annotating the information source include: obtaining the information source text from the target data; extracting the field content of a first preset field from the information source text; wherein the first preset field includes at least the speaker, topic name, and occurrence time; generating structured first information based on multiple first preset fields and their field content; extracting the content of a second preset field from the location information of each knowledge unit, wherein the second preset field includes at least the duration and start time; generating structured second information based on the second preset field and its field content; and merging the first information with the second information of each knowledge unit to form the information source content of the knowledge unit.

13. The method according to claim 2, characterized in that, When performing multi-dimensional annotation on each knowledge unit, the steps for annotation topic classification include: obtaining the knowledge system tree of the professional field to which the original file belongs, wherein the knowledge system tree includes multiple nodes at level one and above from the root node to the leaf node, and the node label of each node includes the node classification topic and classification feature description; obtaining the page text of a first page from the target data from front to back, and classifying the page text of the first page according to the node classification topics in the knowledge system tree to obtain a first classification topic, wherein the first classification topic includes multiple node classification topics arranged in order from superior to subordinate nodes; obtaining the text of each knowledge unit, and classifying the text of each knowledge unit according to the node classification topics in the knowledge system tree to obtain a second classification topic, wherein the second classification topic includes one or more node classification topics arranged in order from superior to subordinate nodes; and concatenating the first classification topic and the second classification topic together according to the node level order in the knowledge system tree to form the topic classification of the knowledge unit.

14. The method according to claim 7 or 8, characterized in that, When performing multi-dimensional annotation on each knowledge unit, the steps for annotation topic classification include: obtaining the professional field and topic name from the information source text; obtaining the knowledge system tree of the professional field, which includes multiple nodes at level one and above from the root node to the leaf node, and the node label of each node includes the node classification topic and the topic classification feature description; semantically matching the topic name with the node classification topics of the knowledge system tree to obtain the first node; obtaining multiple second nodes from the first node to the root node based on the hierarchical relationship of the nodes in the knowledge system tree; arranging the node classification topics of the second nodes and the node classification topics of the first nodes in the order of nodes from superior to subordinate to obtain the first classification topic; obtaining the text of each knowledge unit, classifying each knowledge unit text according to the node classification topics in the knowledge system tree to obtain the second classification topic, which includes one or more node classification topics arranged in the order of nodes from superior to subordinate; and concatenating the first classification topic and the second classification topic together in the order of node levels in the knowledge system tree to form the topic classification of the knowledge unit.

15. The method according to claim 1, characterized in that, When performing multi-dimensional annotation on each knowledge unit, the steps for annotating knowledge types include: obtaining the text of each knowledge unit, and classifying the text of each knowledge unit according to a pre-defined knowledge type classification system to obtain the specific knowledge type.

16. The method according to claim 1, characterized in that, When performing multi-dimensional annotation on each knowledge unit, the steps for annotating keywords include: obtaining the text of each knowledge unit and extracting keywords from the knowledge unit text based on a keyword extraction model.

17. An information annotation system for a professional knowledge base, characterized in that, include: The data acquisition module is configured to acquire raw files for annotation, wherein the raw files are of the type of document, image, audio or video; The preprocessing module is configured to preprocess the original file to obtain the target data; The segmentation module is configured to segment the target data to obtain multiple knowledge units; The system also includes an annotation module, which is configured to perform multi-dimensional annotations on each knowledge unit to obtain corresponding annotation information. The dimensions include information source, topic classification, knowledge type, and keywords.

18. An electronic device comprising a processor and a memory, wherein the memory stores a set of computer program instructions, characterized in that, The information annotation method for the professional knowledge base as described in any one of claims 1-16 is implemented when the processor executes the computer program instruction set on the memory.

19. A computer-readable storage medium, wherein, The computer-readable storage medium stores a set of computer program instructions, characterized in that, when the set of computer program instructions is executed by a processor, it implements the information annotation method of any one of the professional knowledge bases as described in claims 1-16.

20. A computer program product comprising a computer program instruction set, characterized in that, When the computer program instruction set is executed by the processor, it implements the information annotation method for the professional knowledge base as described in any one of claims 1-16.

Images