Training sample generation method and device, equipment, medium and program product

Abstract

The embodiment of the invention provides a training sample generation method and device, equipment, a medium and a program product. The method comprises the following steps: identifying a text data segment from a fixed content file; calling a multi-modal large model based on the text data segment and the fixed content file, and generating directory data corresponding to the fixed content file; wherein the directory data comprises directory entries corresponding to the text data segments; the directory entry is used for indicating the position of a text data segment; arranging the text data segments according to the directory entry to obtain target text data corresponding to the fixed content file; wherein the fixed content file and the target text data form the training sample. According to the training sample generation method and device, the equipment, the medium and the program product, structured processing of the text information contained in the fixed content file can be achieved, the structured degree of the training sample is enhanced, and the training effect of a special large model in the professional field is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of large model technology, and in particular to a method, device, equipment, medium and program product for generating training samples. Background Art

[0002] In the field of artificial intelligence and machine learning, the quality and quantity of training samples directly affect the performance and generalization ability of the model. Especially for professional fields, such as medical, financial, legal and other fields, due to the high degree of professionalism in these fields, in order to improve the output performance of the dedicated large models used in these fields, it is often necessary to use the professional corpus of these fields to perform supervised fine-tuning (SFT) on the general large models.

[0003] Taking the supervised fine-tuning of the medical big model as an example, since the output of the medical big model may have a certain impact on the user's physical health, in order to improve the knowledge accuracy of the output of the medical big model, the training samples of the medical big model are usually generated based on proofread books and treatment guidelines. The above materials are mainly saved in the form of fixed content such as PDF (Portable Document Format) files or paper books. In order to effectively use the above materials, it is necessary to extract the information contained in the PDF files or paper books to obtain text data that is easy to edit by computing devices, and then generate training samples for the medical big model based on the extracted text data.

[0004] However, during the actual training process of the dedicated large model, the R&D personnel found that the training samples generated based on the text data extracted by the existing technology did not have a good training effect on the dedicated large model, and the output performance of the trained dedicated large model was difficult to meet the requirements of the professional field. Summary of the invention

[0005] In view of this, multiple embodiments of the present application are dedicated to providing a method, device, equipment, medium and program product for generating training samples to improve the training effect of dedicated large models in professional fields.

[0006] One embodiment of the present application provides a method for generating a training sample, the method comprising: identifying a text data segment from a fixed content file; calling a multimodal large model based on the text data segment and the fixed content file to generate directory data corresponding to the fixed content file; wherein the directory data comprises a directory entry corresponding to the text data segment; the directory entry is used to indicate the location of the text data segment; the text data segment is arranged according to the directory entry to obtain target text data corresponding to the fixed content file; wherein the fixed content file and the target text data form the training sample.

[0007] One embodiment of the present application provides a device for generating training samples, the device comprising: an identification module, used to identify and obtain text data segments from a fixed content file; a generation module, used to call a multimodal large model based on the text data segment and the fixed content file, and generate directory data corresponding to the fixed content file; wherein the directory data includes directory entries corresponding to the text data segment; the directory entries are used to indicate the positions of the text data segments; an arrangement module, used to arrange the text data segments according to the directory entries to obtain target text data corresponding to the fixed content file; wherein the fixed content file and the target text data form the training sample.

[0008] One embodiment of the present application provides a computer device, which includes a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor to implement the aforementioned method for generating training samples.

[0009] One embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by a processor, the method for generating training samples as described above can be implemented.

[0010] One embodiment of the present application provides a computer program product, which is used to implement the aforementioned method for generating training samples.

[0011] In multiple implementations provided in the present application, a multimodal large model is called based on a fixed content file and a text data segment identified from the fixed content file, and the multimodal large model is used to generate directory data corresponding to the fixed content file, wherein the directory data includes a directory item for indicating the location of the text data segment, and then the text data segment is arranged according to the directory item to obtain the fixed content file and the corresponding target text data, and a training sample is formed by the fixed content file and the corresponding target text data. In this way, the multimodal large model can be used to first generate directory data for hierarchical division of the text information contained in the fixed content file, and then the text data segment corresponding to the text information is arranged according to the directory data, and the training sample is formed by the fixed content file and the target text data with a hierarchical structure, thereby improving the training effect of the special large model for the professional field. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of generating training samples according to a method for generating training samples provided in one embodiment of the present application.

[0013] Figure 2 A schematic diagram of a text information area provided for one embodiment of the present application.

[0014] Figure 3 A flowchart of a method for generating training samples provided in accordance with one embodiment of the present application.

[0015] Figure 4 A module diagram of a training sample generation device provided for one embodiment of the present application.

[0016] Figure 5 A schematic diagram of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0018] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0019] In the related art, large model training samples can be generated based on text data extracted from fixed content files in non-editable formats such as PDF. Specifically, the fixed content files can be parsed using an OCR (Optical Character Recognition) tool or a rule-based parsing algorithm to convert the fixed content files into editable format files, and then the text data can be extracted from the editable format files, and training samples can be constructed based on the extracted text data.

[0020] In the case of supervised fine-tuning of a dedicated large model in a professional field, in order to enhance the dedicated large model's ability to understand and reason about professional domain knowledge and enable the output of the dedicated large model to meet the special needs of the professional field, the fixed content files (such as academic papers, contracts, diagnosis and treatment guidelines, etc.) used as text data sources usually contain rich text information such as font styles, paragraph formats, pictures or tables. These rich text information can help the dedicated large model better understand the semantic logic of the context in the information contained in the fixed content files.

[0021] However, the fixed content file parsing method in the related art is difficult to accurately parse the fixed content file and retain its original layout characteristics, resulting in the special large model trained using the training samples generated on this basis having weak understanding and reasoning capabilities for complex information such as professional terms, charts, formulas, etc.

[0022] In order to further explore the causes of the above problems, the following specific analysis is conducted on the OCR tools and rule-based parsing algorithms used in related technologies.

[0023] OCR tools include layout analysis OCR tools and large model OCR tools.

[0024] The layout analysis OCR tool mainly uses image recognition technology to achieve analysis, and has problems such as poor recognition of rare and ambiguous characters, incoherent text information, difficulty in retaining rich text information, and disordered context order of text data obtained by analysis based on complex typesetting such as columns.

[0025] The large model OCR tool mainly relies on the multimodal large model to achieve parsing, which to some extent solves the problem that the layout analysis OCR tool has poor recognition effect on rare and ambiguous characters and difficulty in retaining rich text information. However, the large model OCR tool has the problem of "AI illusion", that is, the output of the large model OCR tool may contain some additional information generated by the large model, or the output of the large model OCR tool deletes the information of the original fixed content file. In this way, since the additional information has not been manually proofread or reviewed, if the training samples are generated directly based on the output of the large model OCR tool, it may cause the dedicated large model to learn wrong or incomplete knowledge, reducing the accuracy of the knowledge output by the dedicated large model. In addition, for complex layouts such as columns, the parsing results of the large model OCR tool still have the problem of context order disorder. In addition, due to the lack of layout analysis, the large model OCR tool cannot filter invalid information such as headers, footers, authors, and comments that are not closely related to the main content of the fixed content file, resulting in these invalid information being mixed in the text data and the training samples generated based on the text data, making the training of the dedicated large model interfered by invalid information.

[0026] In response to the problems of the above OCR tools, the rule-based parsing algorithm is used to parse fixed content files, which can retain rich text information relatively completely, but the parsing effect for complex layouts such as columns is still poor. In addition, the rule-based parsing algorithm can only parse fixed content files converted from editable format files. For fixed content files composed of scanned images, the rule-based parsing algorithm cannot be used to parse them.

[0027] In addition, in the case where a fixed content file contains a large amount of non-text information such as tables, formulas, and pictures, the text data parsed using the above parsing method will have a problem of context order disorder.

[0028] Therefore, it is necessary to provide a method for generating training samples, calling a multimodal large model based on a fixed content file and a text data segment identified from the fixed content file, and using the multimodal large model to generate directory data corresponding to the fixed content file, wherein the directory data includes a directory item for indicating the location of the text data segment, and then arranging the text data segment according to the directory item to obtain the fixed content file and the corresponding target text data, and forming a training sample through the fixed content file and the corresponding target text data. Thus, the multimodal large model is used to generate directory data for hierarchical division of the text information contained in the fixed content file, and then the text data segments corresponding to the text information are arranged according to the directory data, so as to achieve structural processing of the text information contained in the fixed content file, and the fixed content file and the target text data with a hierarchical structure form a training sample to improve the training effect of the special large model in the professional field.

[0029] In multiple implementations provided in the present application, the method for generating a training sample may be applied to a device for generating a training sample.

[0030] The device for generating training samples can be an electronic device with certain computing power and network access capabilities. The electronic device can be a desktop computer, a laptop computer, a tablet computer, a smart phone, or a server. The electronic device can call the multimodal large model through a network connection to perform the task of generating training samples. Specifically, the electronic device may include a processor, a memory, and a network access module for network communication. The processor can be used to perform tasks such as data recognition, calling the multimodal large model, and text arrangement in the process of generating training samples; the memory can be used to store received fixed content files, identified text data segments, generated directory data, and formed training samples; the network access module can be used to connect to a remotely deployed multimodal large model server to ensure that data transmission and calling of the multimodal large model can be carried out efficiently and stably.

[0031] See also Figures 1 to 3 . One embodiment of the present application provides an application scenario example of a training sample generation device. The training sample generation device can generate training samples for training a special large model in a professional field by calling a multimodal large model to perform structured processing on the text information contained in the input fixed content file. In this scenario example, taking medical research papers in the field of diabetes management as fixed content files to generate training samples for training a medical large model as an example, the process of generating training samples by the training sample generation device is explained.

[0032] The medical research paper is stored in PDF format and includes multiple parts such as abstract, introduction, research methods, research results, discussion of results and conclusion. The medical research paper includes 10 content pages, each of which contains different types of text data segments.

[0033] The user submits the medical research paper to a training sample generating device.

[0034] After receiving the medical research paper, the recognition module in the training sample generation device may first perform the step of recognizing and obtaining text data segments from the medical research paper. Figure 3 The recognition module can use content tagging technology to divide the medical research paper into different text information areas. Each text information area can have a label. The label can show the type and confidence of the text information area. For example, Figure 3 The label 5 in is "Main text 0.98", that is, the information in the text information area corresponding to label 5 has a 98% probability of being the main text of the medical research paper. After completing the division of the text information area, the recognition module can perform text recognition on each text information area and extract the text data segment corresponding to each text information area, and the type of the text data segment is the same as the type of the text information area. For example, in a text information area of ​​type title, the recognition module can recognize the title text data segment "Causes and management methods of diabetes"; in a text information area of ​​type text, the recognition module can recognize the main text data segment, which is a detailed description of the design and implementation process of the clinical trial.

[0035] After obtaining the text data segment corresponding to each text information area, the step of generating directory data can be performed by the generation module in the training sample generation device. The directory data may include directory entries corresponding to the text data segments, and the directory entries may be used to indicate the specific location of the text data segments in the fixed content file. For example, the directory entry may indicate that the title text data segment is located on page 1 of the fixed content file, and indicate that the body text data segment is located on page 4 of the fixed content file. The execution process of the step of generating directory data is as follows.

[0036] First, for a single content page, the generation module can merge text data segments with continuous positions and the same type according to the paragraph arrangement direction and the type of text data segments in the content page. For example, in the case where a single content page is arranged in columns, if there are two text data segments of the main text located in the same column, the generation module can merge the two text data segments into one text data segment to reduce the risk of semantic order disorder in the subsequent arrangement of the text data segments. In addition, the generation module can also generate a vector representation of the first sentence of each text data segment and a vector representation of the last sentence of the paragraph. By matching the vector of the last sentence of the paragraph with the first sentence vector of the adjacent paragraph, the text data segments with continuous contextual relationships are identified for merging. For example, after obtaining two text data segments belonging to the same content page, the last sentence of one text data segment is "The results of this study show that comprehensive management significantly reduces the blood sugar level of patients." The first sentence of the other text data segment is "These results further verify the effectiveness of the comprehensive management strategy." After generating the vector representations of the two sentences, it can be determined through vector matching that the two sentences have a semantic continuity relationship, and the two text data segments can be merged into one text data segment.

[0037] Secondly, for adjacent content pages, the generation module can merge text data segments through vector matching. For example, two text data segments belonging to the 5th and 6th content pages are obtained. The last sentence of the text data segment on the 5th content page is "Therefore, comprehensive management of diabetes requires multidisciplinary collaboration." The first sentence of the text data segment on the 6th content page is "This study further explored the effectiveness of different treatment methods." After generating the vector representation of these two sentences, it can be determined through vector matching that there is a semantic continuity relationship between the two sentences, and the two text data segments can be merged into one text data segment, thereby improving the semantic coherence of the training samples and the structural processing effect of the training samples.

[0038] Subsequently, the generation module may call the multimodal big model based on the merged text data segment and the medical research paper, and use the output of the multimodal big model as the directory data corresponding to the medical research paper.

[0039] After the directory data corresponding to the medical research paper is generated using the multimodal big model, the arrangement module in the training sample generation device can arrange the text data segments according to the directory items to obtain the target text data with a hierarchical structure. In addition, the fixed content file and the generated target text data together form the training sample for training the medical big model. For example, after the medical research paper is subjected to the above-mentioned structured processing to obtain the target text data, the training sample generated based on the medical research paper and the corresponding target text data can include multiple parts such as semantically complete and coherent research methods, research results, discussion of results and conclusions, thereby achieving effective use of the information in the medical research paper in the process of training the medical big model and improving the training effect of the medical big model.

[0040] See also Figure 3 One embodiment of the present application provides a method for generating a training sample. The method for generating a training sample may include the following steps.

[0041] Step S110: Identify and obtain text data segments from the fixed content file.

[0042] Step S120: calling the multimodal large model based on the text data segment and the fixed content file to generate directory data corresponding to the fixed content file; wherein the directory data includes directory entries corresponding to the specified text data; and the directory entries are used to indicate the location of the text data segment.

[0043] Step S130: Arranging the text data segments according to the directory items to obtain target text data corresponding to the fixed content file; wherein the fixed content file and the target text data form training samples.

[0044] In this embodiment, since the content in the fixed content file is not editable, in order to improve the utilization rate of the information contained in the fixed content file, the content in the fixed content file can be converted into an editable format through text recognition and extraction technology.

[0045] In this embodiment, the fixed content file may be a file with fixed content and stored in a non-editable format. Specifically, the fixed content file may be stored in PDF or image format, and may include plain text information and rich text information. Among them, the plain text information may be the information contained in the text of the fixed content file, and the rich text information may be the information contained in the content other than text in the fixed content file, for example, information contained in the content such as font style, paragraph format, page layout, tables and pictures. During the model training process, rich text information helps to strengthen the model's understanding of the semantics of the training samples. For example, bold fonts can emphasize key points, and different paragraph indentation levels can represent different levels of semantics.

[0046] In this embodiment, the text data segment can be a data unit identified and segmented from a fixed content file. Specifically, the text data segment includes plain text information and rich text information. In a text data segment, the semantic logic contained in the plain text information has a certain degree of integrity. Taking a medical research paper stored in PDF format as an example of a fixed content file, the text data segment can be the title of the medical research paper, some paragraphs of the text, tables, pictures, etc.

[0047] In this embodiment, after multiple text data segments are obtained by identifying fixed content files, in order to improve the utilization rate of information contained in the fixed content files, a multimodal large model can be used to implement hierarchical combing of the fixed content files.

[0048] In this embodiment, the multimodal large model can be a deep learning model that can process and understand multiple modal information such as text and images. Specifically, the multimodal large model can simultaneously process and understand input information of different modalities, and use its semantic understanding ability and semantic generation ability to output comprehensive information as specified.

[0049] In this implementation, the directory data may be a structured index of information included in the fixed content file.

[0050] In this embodiment, the directory item can be a specific entry in the directory data corresponding to a specific text data segment. Specifically, each directory item can indicate the starting position, ending position and hierarchical level of the corresponding text data segment in the fixed content file. Taking the medical imaging report stored in PDF format as an example, the directory data and directory items are as follows:

[0051] 1. Basic information of the patient

[0052] 2. Inspection Overview

[0053] 3. Image Description

[0054] 3.1 Chest CT scan results

[0055] 3.2 Abdominal B-mode ultrasound imaging results

[0056] 4. Diagnostic opinion

[0057] 5. Suggestions and precautions

[0058] 6. Doctor’s signature and date

[0059] In this embodiment, the multimodal large model is called based on the text data segment and the fixed content file to generate the directory data corresponding to the fixed content file. The text data segment, the fixed content file and the prompt instruction for indicating the output form of the multimodal large model can be input into the multimodal large model, so that the multimodal large model can output the directory data corresponding to the fixed content file in the form required by the prompt instruction. Continuing to take the medical imaging report whose fixed content file is stored in PDF format as an example, the text data segment obtained by identifying the medical imaging report may include: a plain text data segment, that is, text explaining the medical imaging, and a rich text data segment, that is, the font style and paragraph format of the medical imaging picture and the text of the explanation. The plain text data segment, the rich text data segment, the medical imaging report and the prompt instruction for instructing the multimodal large model to output the directory in the specified format can be input into the multimodal large model together, and the output of the multimodal large model is used as the directory data corresponding to the medical imaging report. Among them, the prompt instruction may be "As a professional medical imaging report analysis expert, please generate a directory data that can accurately reflect the structure and content of the medical imaging report based on the content of the medical imaging report. The directory data should include but not be limited to the following aspects:..."

[0060] In this embodiment, after the directory data is generated using the multimodal large model, the text data segments can be inserted into the directory data according to the correspondence between the directory items in the directory data and the text data segments to obtain the target text data.

[0061] In this embodiment, the text data segments are arranged according to the directory items. The sorting model can be used to arrange the multiple text data segments in a logical order and hierarchical structure based on the position of the corresponding text data segments indicated by the directory items in the fixed content file to form structured target text data. For example, the target text data can be in markdown format. The target text data retains the complete semantics and hierarchical relationship of the information contained in the fixed content file, which improves the structuring degree of the training sample and is conducive to improving the model's understanding of the semantics in the training sample.

[0062] In the implementation mode of the application, the multimodal big model is called based on the fixed content file and the text data segment identified from the fixed content file, and the directory data corresponding to the fixed content file is generated by the multimodal big model, wherein the directory data includes a directory item for indicating the location of the text data segment, and then the text data segment is arranged according to the directory item to obtain the fixed content file and the corresponding target text data, and the training sample is formed by the fixed content file and the corresponding target text data. The directory data for hierarchical division of the text information contained in the fixed content file is generated by the multimodal big model, and the text data segment corresponding to the text information is arranged according to the directory data, so as to achieve the structured processing of the text information contained in the fixed content file, enhance the structured degree of the training sample, and thus improve the training effect of the special big model for the professional field.

[0063] In order to improve the segmentation effect of text data segments and improve the integrity of structured information in the identified text data segments, in some embodiments, the training sample generation device can mark the content of the fixed content file to obtain a text information area; perform text recognition on the text information area to obtain a text data segment corresponding to the text information area.

[0064] In this embodiment, content tagging can be to identify and tag different types of content in a fixed content file using a pre-trained file segmentation model. Specifically, the content types in the fixed content file may include headers and footers, titles, authors, body text, tables, pictures, etc. Content tags can be displayed in the form of tags. For example, tags may include content types and confidence levels. Among them, content types may include titles, body text, tables, pictures, references, notes, etc. Confidence levels may indicate the possibility that the content in the text information area belongs to a certain content type.

[0065] In this embodiment, the text information area can be an area containing a specific type of content identified during the content marking process. Specifically, a two-dimensional coordinate system can be established for each content page of the fixed content file, and the text information area can be determined based on at least two coordinate points in the two-dimensional coordinate system.

[0066] In this embodiment, text recognition can be to use an OCR tool or a rule-based parsing algorithm to identify the content in the text information area. Specifically, text recognition may include steps such as character detection, character segmentation, and character recognition. The content obtained by text recognition may include the following three categories. The first category: invalid content such as headers and footers; the second category: rich text content such as pictures, tables, and formulas; the third category: plain text content such as titles and text. Text recognition can directly filter invalid content, that is, invalid content will not appear in the text data segment, and can convert the format of rich text content into a text data segment in a specific format, for example, a text data segment in markdown format, without performing special processing on plain text content.

[0067] Taking the Yolo V8 model as a file segmentation model to identify text data segments from medical research papers related to diabetes management stored in PDF format as an example, the process of identifying text data segments is explained.

[0068] First, the Yolo V8 model can be pre-trained using annotated and reviewed PDF files related to diabetes management to improve the Yolo V8 model's ability to segment PDF files in the field of diabetes management.

[0069] Subsequently, the medical research paper can be input into the pre-trained Yolo V8 model, and the Yolo V8 model can be used to mark the content of the medical research paper. Specifically, the medical research paper can have multiple content pages, and the Yolo V8 model can establish a two-dimensional coordinate system for each content page of the medical research paper, identify and segment the different types of content contained in each content page, and output the coordinates of the four vertices of each text information area in the two-dimensional coordinate system, as well as the content type and confidence of each text information area.

[0070] Next, the position of each text information area in a single content page can be determined based on the coordinates of the diagonal vertices of each text information area, and a label corresponding to each text information area is formed based on the content type and confidence of each text information area, for example, the paper title area, the main text area, and the reference area.

[0071] Finally, the layout analysis OCR tool or the large model OCR tool can be used to identify and extract the plain text information and rich text information in each text information area to obtain the text data segment corresponding to each text information area. For example, the text data segment corresponding to the title area of ​​the paper can be "Diabetes management model and prevention and control of diabetic retinopathy".

[0072] In the actual training sample generation process, the researchers found that when the fixed content file is laid out in a complex layout such as columns, if the multimodal large model is called and the text data segments are sorted directly based on the text data segments identified from the fixed content file, there are still many contextual semantic order disorders in the target text data obtained in the end. The cause of this problem was explored and it was found that the area range of each text information area directly identified is small, and accordingly, the overall number of text data segments is large, while the amount of data in each text data segment is small, and the semantics of some text data segments are incomplete, so it is easy to cause confusion in the process of arranging a large number of text data segments.

[0073] In some embodiments, in order to reduce the overall number of text data segments, reduce the risk of semantic order disorder in the subsequent arrangement of text data segments, and improve the semantic coherence in the training samples, the training sample generation device can merge multiple text data segments along the paragraph arrangement direction of the fixed content file when multiple text data segments with consecutive positions are of the same type to obtain a merged text data segment.

[0074] In this embodiment, the text data segment may include multiple types. Specifically, the type of the text data segment may be the same as the content type of the corresponding text information area. For example, the type of the text data segment obtained by performing text recognition on the paper title area may be the paper title.

[0075] In this embodiment, multiple text data segments of continuous position and the same type are merged along the paragraph arrangement direction of the fixed content file, and this can be performed for a single content page of the fixed content file. Specifically, the content page of the fixed content file can be a page constituting the fixed content file, and each content page may contain text data segments of different types. The continuous position of multiple text data segments can be the continuous coordinates of the text information areas corresponding to the text data segments in the two-dimensional coordinate system of the single content page. For example, the homepage of a medical research paper is typeset in a columnar manner. By comparing the coordinates of the text information areas corresponding to two text data segments of the type both being body text, the two text information areas are located in adjacent positions of the same column along the paragraph arrangement direction of the homepage. For example, if the difference in the coordinate widths of the two text information areas is within 10% of the overall width of the content page, it can be determined that the two text data segments are continuous in position and of the same type, and the two text data segments are merged into one text data segment.

[0076] In some embodiments, researchers found that in some cases, although the number of text data segments has been reduced after merging, the semantic logic of some text data segments is still discontinuous when generating training samples based on the text data segments merged using the above method. Taking the recognition of medical records stored in PDF format as an example, the patient's diagnosis conclusion text data segment may be closely related to the subsequent treatment plan text data segment in terms of semantic logic. However, the diagnosis conclusion text data segment and the treatment plan text data segment cannot be merged due to their different types.

[0077] In order to improve the logical relevance of semantics in training samples and further reduce the overall number of text data segments, in some embodiments, the training sample generation device can generate a segment-first sentence vector representation of the segment-first sentence and a segment-last sentence vector representation of the segment-last sentence in the text data segment; perform vector matching on the segment-first sentence vector representation of the text data segment and the segment-last sentence vector representation of another text data segment to obtain a text data segment with a continuous contextual relationship; merge the text data segments with a continuous contextual relationship to obtain a merged text data segment. Thus, it is possible to determine whether different text data segments have semantic continuity through the vector representation of two sentences that may have semantic continuity in different text data segments, and merge the text data segments with a continuous contextual relationship, thereby providing more complete contextual information for model training.

[0078] In this embodiment, the vector representation of the first sentence and the vector representation of the last sentence of the paragraph can be used to represent the semantic features of the first sentence and the last sentence of the text data segment. Specifically, the first sentence and the last sentence of the text data segment can be converted into a numerical vector form by feature extraction and vectorization.

[0079] In this embodiment, the vector matching between the vector representation of the sentence at the beginning of the paragraph and the vector representation of the sentence at the end of the paragraph can be achieved by calculating the vector similarity between the two sentence vector representations. Specifically, the vector similarity can be calculated using methods such as cosine similarity and Euclidean distance. When the vector similarity is less than a specified threshold, it can be determined that there is a semantic connection between the sentence at the beginning of the paragraph and the sentence at the end of the paragraph, and further, it can be determined that there is a continuous contextual relationship between the two text data segments. Among them, the specified threshold can be determined based on the semantic content.

[0080] In some embodiments, since a fixed content file may include multiple content pages, and text data segments with continuous contextual relationships may be located on the same content page of the fixed content file. Therefore, in some embodiments, the training sample generation device may obtain text data segments belonging to the same content page; perform vector matching on the first sentence vector representation of the first sentence of the text data segment and the last sentence vector representation of other text data segments in the same content page, and obtain text data segments with continuous contextual relationships, so that the vector matching process is only performed on the text data segments in the same content page, reducing the waste of computing resources and the risk of semantic rupture caused by cross-page vector matching, and improving the overall quality of the training samples. For example, taking the identification of medical records stored in PDF format as an example, the patient's diagnosis conclusion text data segment may be closely related to the subsequent treatment plan text data segment in semantic logic. When the diagnosis conclusion text data segment and the treatment plan text data segment are located on the same content page, the diagnosis conclusion text data segment and the treatment plan text data segment can be merged by matching the last sentence vector representation of the diagnosis conclusion text data segment with the first sentence vector representation of the treatment plan text data segment.

[0081] In some cases, text data segments with continuous contextual relationships may also be located in adjacent content pages of a fixed content file. Taking the identification of a medical imaging report stored in PDF format as an example, the medical imaging report contains multiple content pages, and different content pages contain text descriptions and related diagnostic descriptions of different medical images. If a medical image is located at the end of a content page, and the text description and diagnostic description of the medical image are located at the beginning of the next content page, then the two text data segments with continuous contextual relationships are located in different content pages. In order to improve the semantic coherence between text data segments located in different content pages and further improve the quality of training samples, in some embodiments, the training sample generation device can also match the end sentence vector of the text data segment at the end of the previous content page in two adjacent content pages with the beginning sentence vector of the text data segment at the beginning of the next content page, and obtain the text data segment with continuous contextual relationships.

[0082] By matching the vectors based on the types of different text data segments, their positions within the same content page, and the vector representations of the first and last sentences of different text data segments, not only can text data segments of the same type and continuous positions be identified and merged, but also text data segments with contextual coherent semantics can be merged based on the semantic associations between sentences. This improves the semantic integrity and structured degree of training samples for fixed content files with complex layouts, thereby improving the understanding depth and reasoning accuracy of the dedicated large model.

[0083] See also Figure 4The embodiment of the present application also provides a training sample generation device, including: a recognition module, a generation module and an arrangement module.

[0084] The recognition module can be used to recognize text data segments from fixed content files.

[0085] The generation module can be used to call the multimodal macro model based on the text data segment and the fixed content file to generate directory data corresponding to the fixed content file. The directory data can include directory items corresponding to the specified text data. The directory items can be used to indicate the location of the text data segment.

[0086] The arrangement module can be used to arrange the text data segments according to the directory items to obtain the target text data corresponding to the fixed content file, wherein the fixed content file and the target text data form training samples.

[0087] The specific functions and effects achieved by the training sample generation device can be explained with reference to other embodiments of the present application, and will not be repeated here. The various modules in the training sample generation device can be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0088] The embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor implements the aforementioned method for generating training samples.

[0089] The embodiment of the present application also provides a computer program product including instructions, which, when executed by a processor, enables a computer to implement the aforementioned method for generating training samples.

[0090] See also Figure 5 The present embodiment may provide a computer device, the computer device comprising: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, the instructions are executed by the one or more processors, so that the one or more processors implement the aforementioned method for generating training samples.

[0091] In some embodiments, the computer device may include a processor, a non-volatile storage medium, an internal memory, a communication interface, a display device, and an input device connected by a system bus. The non-volatile storage medium may store an operating system and related computer programs.

[0092] The user information or user account information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, etc.) involved in multiple implementation methods of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0093] It should be understood that the specific examples in this article are only intended to help those skilled in the art to better understand the embodiments of the present application, rather than to limit the scope of the present invention.

[0094] It can be understood that in the various implementations of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of the present application.

[0095] It can be understood that the various embodiments described in this application can be implemented individually or in combination, and the embodiments of this application are not limited to this.

[0096] Unless otherwise stated, all technical and scientific terms used in the embodiments of the present application have the same meaning as those generally understood by those skilled in the art of the technical field of the present application. The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the scope of the present application. The term "and / or" used in the present application includes any and all combinations of one or more related listed items. The singular forms of "a kind of", "above" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0097] It can be understood that the processor of the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method implementation can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0098] It is understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (programmable ROM, PROM), an erasable programmable read-only memory (erasable PROM, EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory may be a random access memory (RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0099] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0100] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method implementation methods and will not be repeated here.

[0101] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0103] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0104] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0105] The above is only a specific implementation of the present application, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for generating training samples, characterized in that: The method comprises: Identify a text data segment from a fixed content file; Based on the text data segment and the fixed content file, the multimodal macro model is called to generate directory data corresponding to the fixed content file; wherein the directory data includes a directory entry corresponding to the text data segment; and the directory entry is used to indicate the location of the text data segment; The text data segments are arranged according to the directory items to obtain target text data corresponding to the fixed content file; wherein the fixed content file and the target text data form the training sample.

2. The method according to claim 1, characterized in that The step of identifying and deriving a text data segment from a fixed content file comprises: Marking the fixed content file to obtain a text information area; Perform text recognition on the text information area to obtain a text data segment corresponding to the text information area.

3. The method according to claim 1, characterized in that The text data segment includes multiple types; the method further includes: In the case where a plurality of text data segments that are consecutively located along the paragraph arrangement direction of the fixed content file are of the same type, the plurality of text data segments are merged to obtain a merged text data segment.

4. The method according to claim 1, characterized in that: The method further comprises: Generate a first sentence vector representation of the first sentence of the text data segment and a last sentence vector representation of the last sentence of the paragraph; Perform vector matching on the sentence vector representation of the beginning of a text data segment and the sentence vector representation of the end of another text data segment to obtain text data segments with continuous contextual relationships; The text data segments having the continuous contextual relationship are merged to obtain merged text data segments.

5. The method according to claim 4, characterized in that The fixed content file includes a plurality of content pages; the step of performing vector matching on a sentence vector representation of a segment head of a text data segment and a sentence vector representation of a segment tail of another text data segment to obtain a text data segment having a continuous contextual relationship includes: Get the text data segments belonging to the same content page; The first sentence vector representation of the first sentence in the text data segment is matched with the last sentence vector representation of other text data segments in the same content page to obtain text data segments with continuous contextual relationships.

6. The method according to claim 5, characterized in that The method further comprises: In two adjacent content pages, the end sentence vector of the text data segment at the end of the first content page is matched with the beginning sentence vector of the text data segment at the beginning of the second content page to obtain text data segments with continuous contextual relationships.

7. A training sample generation device, characterized in that: include: A recognition module, used for recognizing and obtaining text data segments from a fixed content file; A generation module, configured to call the multimodal macro model based on the text data segment and the fixed content file to generate directory data corresponding to the fixed content file; wherein the directory data includes a directory entry corresponding to the text data segment; and the directory entry is used to indicate a location of the text data segment; An arrangement module is used to arrange the text data segments according to the directory items to obtain target text data corresponding to the fixed content file; wherein the fixed content file and the target text data form the training sample.

8. A computer device, characterized in that: The computer device includes a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the method for generating training samples according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and when the at least one computer program is executed by a processor, the method for generating training samples according to any one of claims 1 to 6 can be implemented.

10. A computer program product, characterized in that The computer program product is used to implement the method for generating training samples according to any one of claims 1 to 6.

Images