A battery document image differentiation description generation method and system

Abstract

The application provides a battery literature image differentiation description generation method and system, which comprises the following steps: obtaining and scanning the battery literature, and then obtaining the region through a preset algorithm; each chart is assigned a unique identifier and a mapping table is constructed based on the document coordinates of the chart in the battery literature; a classification prompt word is input into a pre-trained multi-modal large model to output a chart type and assign a processing routing label, and after obtaining the classification metadata through analysis, differentiation processing is performed; context data packets are obtained by directly describing the text and combining the associated text, and after processing, semantic enhanced context data packets are obtained, and a prompt word template is called to ask differentiated questions; target prompt words are generated by inserting the description sample and the semantic enhanced context data packet into the prompt word template, and natural language description text is obtained by inputting the chart type into the pre-trained multi-modal large model. The application can reduce the term cognitive bias and context semantic loss during the generation of natural language description of battery literature charts, and improve the reliability of the natural language description text.

Description

Technical Field

[0001] This invention relates to the field of data annotation, and in particular to a method and system for generating differentiated descriptions of battery literature images. Background Technology

[0002] Existing technologies for document parsing and image understanding in the battery field mostly adopt an end-to-end processing architecture based on a general multimodal large model. First, a general optical character recognition (OCR) engine or document parsing tool is used to scan the entire PDF document to extract the image regions and corresponding text information. Then, the extracted images and nearby text fragments are input into the encoder of a pre-trained multimodal model to align visual and linguistic features. A unified, unstructured prompt word template is used to drive the model to generate natural language descriptions or classification results for the images.

[0003] Existing technologies have significant shortcomings when applied to professional literature processing in the battery field. First, general multimodal models lack specialized training for battery materials science, resulting in weak perception of specific fine-grained entities and complex chemical symbols, and a lack of accuracy in domain entity recognition and terminology standardization. Second, existing technologies use uniform prompt templates to process all charts, failing to establish a multi-level fine-grained classification system. The generated descriptions often omit key data trends or structural details and lack a differentiated description mechanism for chart types. Furthermore, treating extracted charts as independent visual units fails to effectively integrate the contextual clues of the charts in the original document, resulting in generated descriptions that remain only at the visual level and fail to reflect the experimental conclusions or logical connections intended by the authors in the original text. Finally, there is a high dependence on fine-tuning data and low generalization. Furthermore, relying on the construction of large-scale, high-quality labeled data for model fine-tuning generates high data labeling and computational costs, and the model training lacks flexibility and rapid transfer capabilities. Therefore, a method for generating differentiated descriptions of battery literature images is needed to address the issues of terminological cognitive bias and lack of contextual semantics in general multimodal large models within the battery professional field, and to achieve accurate classification, differentiated analysis, and deep semantic description generation of various types of charts in complex literature. Summary of the Invention

[0004] This invention provides a method and system for generating differentiated descriptions of battery literature images. Its main purpose is to solve the problems of terminological cognitive bias and lack of contextual semantics in the generation of battery literature descriptions in the prior art.

[0005] To achieve the above objectives, the present invention provides a method for generating differential descriptions of battery literature images, comprising: After acquiring and scanning battery literature, the system divides the region into text, graphic, and background areas according to pixel distribution using a preset algorithm. A unique identifier is assigned to each chart in the graphic area, and a mapping table is constructed based on the document coordinates of the charts in the battery literature. Classification prompts are constructed based on preset visual feature boundaries, and the target charts in the mapping table are semantically classified using a pre-trained multimodal large model to obtain chart types. Processing route labels are assigned to each target chart type. The chart types include data charts, schematic diagrams, multiple charts, tables, and formulas. The chart type is parsed to obtain category metadata, and the target chart is differentiated according to the route tag; Based on the document coordinates of the target chart in the battery literature, the caption text of the target chart is extracted from the battery literature as the direct description text of the target chart, the citation statement and related context of the target chart are extracted as related text, the direct description text and the related text are combined to obtain a context data package, and the context data package is processed by a preset knowledge dictionary to perform terminology correction and format standardization to obtain a semantically enhanced context data package. According to the processing route label, the corresponding prompt word template is invoked to perform differentiated questioning. A description sample consistent with the chart type of the target chart is retrieved from the preset standard description library and inserted into the prompt word template with the semantic enhancement context data package to generate the target prompt word. The chart type and the target prompt are input into the pre-trained multimodal large model to obtain the natural language description text of the target chart.

[0006] Optionally, the process of acquiring and scanning battery documents and then dividing them into text, graphic, and background regions according to pixel distribution using a preset algorithm includes: A page pixel matrix for identifying battery literature is processed using a pixel density clustering-based layout analysis algorithm to identify text line ranges and mark them as text regions; an image recognition algorithm based on edge detection and contour extraction is processed using the page pixel matrix to locate image ranges and mark them as image regions; and a background color consistency analysis algorithm is processed using the page pixel matrix to separate background ranges that match a preset background color threshold and mark them as background regions.

[0007] Optionally, the classification prompts constructed based on preset visual feature boundaries include: The data graph is a visualization that includes coordinate axes, schematic diagrams, and data geometric elements, including cyclic voltammetry graphs, charge-discharge curves, bar charts, and scatter plots; The schematic diagrams described are schematic diagrams illustrating the microstructure of battery materials, battery module structure, or reaction principle, including scanning electron microscope images, transmission electron microscope images, structural disassembly diagrams, and principle schematic diagrams; The multiple images are combined images where a single chart contains at least two independent sub-images and has sub-image index labels; The table is a data table with an explicit row and column grid structure or an aligned text array; The formula is an independent region containing mathematical derivations, chemical reaction equations, or molecular structural formulas.

[0008] Optionally, the training process for a multimodal large model includes: Obtain a training sample set, which contains multiple samples, each containing a chart from a battery literature document; the multiple samples in the training sample set are labeled with the chart type ground truth and the natural language description ground truth. An initial multimodal large model is obtained, comprising a visual encoder, a text encoder, a multimodal fusion layer, a classifier, and a language decoder. The visual encoder takes samples from the training sample set as input and outputs image view features. The text encoder takes the target prompt words and semantically enhanced context data packets corresponding to the samples as input and outputs text features. The multimodal fusion layer fuses the image visual features and the text features to obtain target features. The target features are then processed by the classifier to output a chart type prediction value, and the target features are processed by the language decoder to output a natural language description prediction value. A first loss function is constructed based on the deviation between the true value and the predicted value of the chart type. A second loss function is constructed based on the deviation between the true value and the predicted value of the natural language description. A joint loss function is constructed using the first and second loss functions. The parameters of the initial multimodal large model are iteratively updated with the goal of minimizing the joint loss function to obtain the multimodal large model.

[0009] Optionally, differentiating the target chart based on the route label includes: When processing data graphs, a data trend focus mode is used to identify axis ranges, curve change trends, peak voltage data characteristics, and current data characteristics. When processing the schematic diagram, a principle-structure focus mode is used to identify micro-textures, component spatial relationships, and physical component names.

[0010] When processing multiple graphs, a subgraph relationship attention mode is adopted to trigger the subgraph segmentation algorithm to identify the subgraph index label position, decompose the combined graph into independent subgraphs, and recursively call the classification process of this stage; A dedicated formatting mode is used when processing tables or formulas to generate structured data through an optical character recognition engine and formula code through a formula recognition engine.

[0011] Optionally, the process of performing terminology correction and format standardization on the context data packet using a preset knowledge dictionary includes: The preset knowledge dictionary is obtained; the preset knowledge dictionary includes chemical formulas of battery materials, battery-related terminology, and standard unit symbols; The context data packet is segmented to obtain data lexical units, and the edit distance between the lexical units and the preset knowledge dictionary is calculated; when the edit distance is lower than a preset threshold, the words in the context data packet are replaced with standard terms in the preset knowledge dictionary; The subscript and superscript format in the context data packet is standardized using regular expression correction rules.

[0012] Optionally, the method further includes: storing the chart type, the target prompt word, and the natural language description as a log and saving it to a cloud storage device.

[0013] To address the aforementioned problems, the present invention also provides a battery literature image differential description generation system. This system is used to execute the aforementioned battery literature image differential description generation method, and includes: The region division module is used to acquire and scan battery literature and then divide it into text region, graphic region and background region according to pixel distribution using a preset algorithm. It assigns a unique identifier to each chart in the graphic region and constructs a mapping table based on the document coordinates of the chart in the battery literature. The classification processing module is used to construct classification prompts based on preset visual feature boundaries, and to perform semantic classification on the target charts in the mapping table to obtain chart types through a pre-trained multimodal large model, and to assign processing route labels to each target chart type; the chart types include data charts, schematic diagrams, multiple charts, tables and formulas; the module parses the chart types to obtain classification metadata, and performs differentiated processing on the target charts according to the route labels; The target prompt word generation module is used to extract the caption text of the target chart as the direct description text of the target chart based on the document coordinates of the target chart in the battery literature, extract the citation statements and related context of the target chart as related text, combine the direct description text and the related text to obtain a context data package, and perform terminology correction and format standardization processing on the context data package through a preset knowledge dictionary to obtain a semantically enhanced context data package; call the corresponding prompt word template according to the processing routing label to perform differentiated questioning, retrieve the description sample with the same chart type as the target chart from the preset standard description library and insert it into the prompt word template with the semantically enhanced context data package to generate the target prompt word; The natural language description output module is used to input the chart type and the target prompt word into the pre-trained multimodal large model to obtain the natural language description text of the target chart.

[0014] To address the above problems, the present invention also provides an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory has a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the battery document image differential description generation method described above.

[0015] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements any of the above-described battery document image differential description generation methods.

[0016] The beneficial effects of this invention are as follows: This invention provides a method and system for generating differentiated descriptions of battery literature images. By dividing battery literature into regions and constructing a mapping table based on document coordinates, it determines the caption text, cited statements, and related context, solving the problem of semantic disconnect between figures and context. This makes the generated descriptions more aligned with the core ideas of the literature, significantly reducing model illusions. It constructs classification prompts and uses a multimodal large model to categorize figure types into data graphs, schematic diagrams, multiple graphs, tables, and formulas, and performs differentiated processing for each type. It corrects and processes context data packages through a pre-set knowledge dictionary, improving the credibility and usability of natural language description text. Through semantically enhanced context data packages, prompt templates, target prompts, and semantically enhanced context data packages, it reduces the need for large-scale annotation data, lowering computational power and data annotation costs. It also reduces terminology cognitive bias and contextual semantic gaps during battery literature description generation.

[0017] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0018] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of a battery literature image differential description generation method according to an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of a battery literature image differential description generation system according to an embodiment of the present invention.

[0021] Figure 3 This is a schematic diagram of the structure of a battery literature image differential description generation method according to an embodiment of the present invention.

[0022] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0024] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0025] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0026] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0027] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0028] Figure 1 This is a flowchart illustrating a method for generating differential descriptions of battery literature images according to an embodiment of the present invention. This application provides a method for generating differential descriptions of battery literature images, which includes the following steps S101-S106: Step S101: After acquiring and scanning the battery literature, the data is divided into text area, graphic area and background area according to pixel distribution using a preset algorithm. A unique identifier is assigned to each chart in the graphic area and a mapping table is constructed based on the document coordinates of the chart in the battery literature.

[0029] Step S102: Construct classification prompt words based on preset visual feature boundaries, and use a pre-trained multimodal large model to perform semantic classification on the target charts in the mapping table to obtain the chart type, and assign processing route labels to each target chart type; the chart types include data charts, schematic diagrams, multiple charts, tables and formulas.

[0030] Step S103: Parse the chart type to obtain category metadata, and perform differentiated processing on the target chart according to the route label.

[0031] Step S104: Based on the document coordinates of the target chart in the battery literature, extract the caption text of the target chart from the battery literature as the direct description text of the target chart, extract the citation statements and related context of the target chart as related text, combine the direct description text and related text to obtain a context data package, and obtain a semantically enhanced context data package after performing terminology correction and format standardization processing on the context data package through a preset knowledge dictionary.

[0032] Step S105: Based on the processing route label, call the corresponding prompt word template to perform differentiated questioning, retrieve the description sample that matches the chart type of the target chart from the preset standard description library, and insert it into the prompt word template with the semantic enhancement context data packet to generate the target prompt word.

[0033] Step S106: Input the chart type and target cue words into the pre-trained multimodal large model to obtain the natural language description text of the target chart.

[0034] In step S101, the battery literature is fully scanned using a deep learning-based document parsing tool, including MinerU and similar page layout analysis models. Each chart has a unique identifier. A mapping table is constructed using the unique identifier of the chart and the document coordinates of the chart in the corresponding battery literature. The document coordinates include the page number and the bounding box coordinates. The bounding box coordinates include the coordinates of the four corners of the minimum bounding rectangle of the chart.

[0035] In this embodiment of the invention, the process of acquiring and scanning battery documents and then dividing them into text, graphic, and background regions according to pixel distribution using a preset algorithm includes: processing the page pixel matrix of the battery documents using a layout analysis algorithm based on pixel density clustering, identifying text line ranges and marking them as text regions; processing the page pixel matrix using a graphic recognition algorithm based on edge detection and contour extraction, locating image ranges and marking them as graphic regions; and processing the page pixel matrix using a background color consistency analysis algorithm, separating background ranges that match a preset background color threshold and marking them as background regions. The background color consistency analysis algorithm uses the Otsu's algorithm (maximum inter-class variance algorithm). A multi-level algorithm combining the layout analysis algorithm based on pixel density clustering, the graphic recognition algorithm based on edge detection and contour extraction, and the background color consistency analysis algorithm is used to segment the text, image, and background regions, improving the accuracy of each region's location.

[0036] In step S102, the constructed classification prompts are input into the pre-trained multimodal large model, which outputs the semantic classification results of the target charts in the mapping table and assigns a unique processing route label to each type of chart.

[0037] In this embodiment of the invention, the training process of the multimodal large model includes steps S1021 to S1023: Step S1021: Obtain the training sample set, which contains multiple samples, each containing a graph from a battery literature; the multiple samples in the training sample set are labeled with the graph type ground truth and the natural language description ground truth.

[0038] Step S1022: Obtain the initial multimodal large model, which includes a visual encoder, a text encoder, a multimodal fusion layer, a classification head, and a language decoder. The visual encoder takes samples from the training sample set as input and outputs image view features. The text encoder takes the target prompt words and semantically enhanced context data packets corresponding to the samples as input and outputs text features. The multimodal fusion layer fuses the image visual features and text features to obtain target features. The target features are processed by the classification head to output chart type prediction values, and the target features are processed by the language decoder to output natural language description prediction values.

[0039] Step S1023: Construct a first loss function based on the deviation between the true value and the predicted value of the chart type, construct a second loss function based on the deviation between the true value and the predicted value of the natural language description, construct a joint loss function through the first and second loss functions, and perform iterative parameter updates on the initial multimodal large model with the goal of minimizing the joint loss function to obtain the multimodal large model.

[0040] Specifically, the visual encoder uses a Vision Transformer; the text encoder uses a Bidirectional Transformer (BERT); the multimodal fusion layer uses a cross-attention fusion module; the classification head includes a first fully connected layer, an activation layer, a second fully connected layer, and a Softmax activation layer; the language decoder uses a GPT-2 decoder; the first loss function uses the first cross-entropy loss function, and the second loss function uses the second cross-entropy loss function; the expression for the first loss function is... The expression for the second loss function is: The expression for the joint loss is: ;in, Indicates the total number of training samples; Indicates the first The true value of the chart type for each sample; Indicates the first Chart type prediction values ​​for each sample; Indicates the first The true value of the natural language description of each sample; Indicates the first Natural language description prediction value for each sample; and This represents the weighting coefficient.

[0041] Furthermore, the data graph includes quantitative data trends, the schematic diagram includes qualitative structural information, and multiple graphs include composite information; in this embodiment of the invention, constructing classification prompts based on preset visual feature boundaries includes: Data graphs are visualizations that include coordinate axes, schematic diagrams, and data geometric elements, including cyclic voltammetry plots, charge-discharge curves, bar charts, and scatter plots.

[0042] The schematic diagrams are illustrations of the microstructure of battery materials, battery module structure, or reaction principle, including scanning electron microscope images, transmission electron microscope images, structural disassembly diagrams, and principle schematic diagrams.

[0043] Multiple charts are combined images that contain at least two independent sub-charts within a single chart and have sub-chart index labels.

[0044] The table is a data table with an explicit row and column grid structure or an aligned array of text.

[0045] The formula is an independent region containing mathematical derivations, chemical reaction equations, or molecular structural formulas.

[0046] In step S103, the chart type of the classification result is parsed to obtain JSON-formatted classification metadata, and differentiated routing processing is performed based on the processing route tags of the chart type. In this embodiment of the invention, differentiated processing of the target chart based on the routing tags includes: When processing data graphs, a data trend focus mode is used to identify axis ranges, curve change trends, peak voltage data characteristics, and current data characteristics.

[0047] When processing the schematic diagram, a principle-structure focus mode is used to identify micro-textures, component spatial relationships, and physical component names.

[0048] When processing multiple graphs, a subgraph relationship attention mode is adopted to trigger the subgraph segmentation algorithm to identify the subgraph index label position, decompose the combined graph into independent subgraphs, and recursively call the classification process of this stage; A dedicated formatting mode is used when processing tables or formulas to generate structured data through an optical character recognition engine and formula code through a formula recognition engine.

[0049] Specifically, micro-texture includes particle size and porosity, and component spatial relationships include cladding layers and core-shell structures; tables or formulas do not generate natural language descriptions, but are directly routed to an optical character recognition (OCR) table restoration engine or a LaTeX formula recognition engine to be converted into structured data or formula code.

[0050] In step S104, text paragraphs starting with common identifiers are retrieved within a preset geometric range of the target chart based on document coordinates as the caption text for the target chart. Common identifiers include Fig, Figure, and Scheme. The content of the question text is extracted as the direct descriptive text of the target chart. Further, battery literature is searched using a preset regular expression to match citation statements containing the target chart number. The sentences before and after the citation statement form the associated context. The citation statement and the associated context are combined to obtain the associated text. The direct descriptive text of each chart is combined with the associated text to obtain the context data package. The preset knowledge dictionary is a battery field knowledge dictionary including chemical formulas of battery materials, battery field terminology, and standard unit symbols. In this embodiment of the invention, the process of performing terminology correction and format standardization processing on the context data package using the preset knowledge dictionary includes steps S1041~S1043: Step S1041: Obtain a preset knowledge dictionary; the preset knowledge dictionary includes chemical formulas of battery materials, battery-related terminology, and standard unit symbols.

[0051] Step S1042: Perform word segmentation on the context data packet to obtain data lexical units, and calculate the edit distance between the lexical units and the preset knowledge dictionary; when the edit distance is lower than the preset threshold, replace the words in the context data packet with standard terms in the preset knowledge dictionary.

[0052] Step S1043: Standardize the subscript and superscript format in the context data packet using regular expression correction rules.

[0053] Specifically, after segmenting the context data packet, the edit distance between the word and the preset knowledge dictionary is calculated. The edit distance is used to represent the degree of difference between two strings. The minimum number of single-character editing operations required to convert one string into another is calculated. Words below the preset threshold are suspected misspellings or variants. The suspected misspellings are replaced with the standard spellings in the preset knowledge dictionary.

[0054] In steps S105 and S106, when calling the corresponding prompt word templates for differentiated questioning, the prompt word template for data charts focuses on data mining, and can include prompts such as "Please describe the physical meaning and units of the X and Y axes in context, analyze the trend of the curve with voltage / time, and clearly point out the key peak data in the figure and the physical meaning they represent; key peak data includes, but is not limited to, redox peak potentials"; the prompt word template for illustrations focuses on structural analysis, and can include prompts such as "Please identify the core components in the figure and describe..." Its spatial relationship and microscopic morphological characteristics are described, and its working mechanism is explained in conjunction with the caption; the core components include the positive electrode, the negative electrode, and the separator, the spatial relationship includes coating and layered stacking, and the microscopic morphological characteristics include spherical particles and cracks; the prompt word template for multi-images focuses on logical association, and the prompt word template can be "This is a set of comparative experimental images. Please describe the contents of sub-images (a) and (b) respectively, and summarize the performance or structural differences between the two before and after modification in conjunction with the context"; sub-images (a) and (b) are multiple images from battery literature and are not the figure numbers of this application.

[0055] Furthermore, one or two input-output pairs matching the target chart type are retrieved from a pre-defined standard description library as description samples and added to a prompt word template. The prompt word template is also added in the form of tags from a semantically enhanced context data package, and the prompt word template includes the phrase "Please describe based on the visual content of the chart and refer to the experimental conclusions in the following context information, but do not create illusions." After the target prompt word and chart type are input into the multimodal large model, the output is a natural language description text that not only accurately restores the visual information in the chart and integrates the experimental conclusions from battery literature, but also correctly uses professional terminology.

[0056] In this embodiment of the invention, the method further includes: storing the chart type, target prompt words, and natural language description as a log and saving it to a cloud storage device.

[0057] Figure 2This is a schematic diagram of a battery literature image differential description generation system according to an embodiment of the present invention. On the other hand, the present invention also provides a battery literature image differential description generation system, which executes a battery literature image differential description generation method, the system comprising: The region segmentation module is used to acquire and scan battery literature and then divide it into text region, graphic region and background region according to pixel distribution using a preset algorithm. It assigns a unique identifier to each chart in the graphic region and constructs a mapping table based on the chart's document coordinates in the battery literature.

[0058] The classification processing module is used to construct classification prompts based on preset visual feature boundaries, and to perform semantic classification of the target charts in the mapping table to obtain the chart type through a pre-trained multimodal large model, and to assign processing route labels to each target chart type; the chart types include data charts, schematic diagrams, multiple charts, tables and formulas; the module parses the chart types to obtain classification metadata, and performs differentiated processing on the target charts according to the route labels.

[0059] The target prompt word generation module is used to extract the caption text of the target chart from the battery literature based on the document coordinates of the target chart in the battery literature as the direct description text of the target chart, and extract the citations and related context of the target chart as the related text. The direct description text and related text are combined to obtain a context data package. After the context data package is processed by a preset knowledge dictionary for terminology correction and format standardization, a semantically enhanced context data package is obtained. The module calls the corresponding prompt word template according to the processing route label to perform differentiated questioning. It retrieves the description sample with the same chart type as the target chart from the preset standard description library and inserts it into the prompt word template with the semantically enhanced context data package to generate the target prompt word.

[0060] The Natural Language Description Output Module is used to input the chart type and target prompt words into a pre-trained multimodal large model to obtain the natural language description text of the target chart.

[0061] On the other hand, embodiments of the present invention also provide an electronic device, the electronic device including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory has a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute in order to implement the above-described battery document image differential description generation method.

[0062] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described battery document image differential description generation method.

[0063] The present invention will now be described with reference to a specific embodiment: Figure 3 This is a schematic diagram of the battery literature image differential description generation method according to an embodiment of the present invention. The present invention proposes a method and system for generating battery literature image differential descriptions.

[0064] Step 1: Document Parsing and Layout Analysis. Unstructured PDF documents in the battery field are converted into structured data that can be processed by computers, enabling physical analysis of the document layout and precise acquisition of charts and graphs.

[0065] 1.1 Full Document Layout Analysis. A dedicated document parsing tool based on deep learning is used to scan the entire document. This tool includes MinerU and similar layout analysis models. Based on pixel distribution characteristics, the document is divided into text areas, chart areas, and background areas.

[0066] 1.2 Chart Cropping and Coordinate Mapping. While extracting charts, the system simultaneously establishes a chart-document coordinate mapping table. This mapping table records the unique identifier of each chart and its geometric coordinates in the original PDF page. The set coordinates include the page number Page N and the bounding box coordinates [x1, y1, x2, y2].

[0067] Step 2: Multi-granularity chart semantic classification and processing strategy routing. Receive the original chart sequence output from Step 1, and utilize the visual understanding capabilities of a multimodal large model to perform semantic classification of the charts. Based on the characteristics of battery-related literature, the charts are divided into five preset types, and a unique processing routing label is assigned to each type, providing a decision-making basis for subsequent differentiated description generation.

[0068] 2.1 Constructing a Domain-Adaptive Classification Hint Project. The hints explicitly define the visual feature boundaries of five major chart types within the battery domain, forcing the model to ignore non-key text and focus on analyzing the chart's layout structure and semantic category. Classification criteria include: (1) Data graphs: Visual graphs containing clear coordinate axes, schematic diagrams and data geometric elements, including cyclic voltammetry (CV), charge-discharge curves (GCD), bar charts and scatter plots, which mainly carry quantitative data trends.

[0069] (2) Schematic diagram: A schematic diagram or photograph describing the microstructure of the battery material, the structure of the battery module and the reaction principle, including SEM / TEM electron microscope images, structural disassembly diagrams and principle schematic diagrams, focusing on qualitative structural information.

[0070] (3) Multiple plots: A composite image containing two or more independent subplots within a single image, usually with subplot index labels such as (a), (b), etc., carrying composite information.

[0071] (4) Table: A data table with an explicit row and column grid structure or an aligned text array.

[0072] (5) Formula: An independent region containing mathematical derivation, chemical reaction equation or molecular structure formula.

[0073] 2.2 Structured Classification Result Output and Policy Routing. The model's inference results are parsed into standard JSON format metadata, and the corresponding processing branches in the flowchart are activated based on the category labels to achieve differentiated routing.

[0074] (1) For the data graph, activate the data trend focus mode; subsequent processing will focus on identifying key data features such as coordinate axis range, curve change trend (rising / falling), peak voltage / current, etc.

[0075] (2) Needle diagram, activate principle structure attention mode; subsequent processing will focus on identifying the spatial relationship of components including particle size and pore micro-texture, including coating layer and core-shell structure, as well as the name of physical components.

[0076] (3) For multi-images, activate the subgraph relationship attention mode; this is a special processing path. The system will trigger the subgraph segmentation algorithm, identify the (a) and (b) label positions, decompose the combined graph into independent subgraphs, and recursively call the classification process of this stage, while retaining the logical relationship between subgraphs including the comparative experimental relationship.

[0077] (4) For tables / formulas, activate the dedicated formatting processing mode; such charts do not enter the subsequent natural language description generation process, but are directly routed to the OCR table restoration engine or LaTeX formula recognition engine to be converted into structured data or formula code.

[0078] Step 3: Contextual Location and Knowledge Enhancement Based on Page Coordinates. This step aims to address the semantic loss of charts when removed from their document context. Utilizing the chart coordinate information retained in Step 1, it accurately retrieves textual information related to the chart from the original PDF document, including figure captions and text citations. Furthermore, it introduces a battery-specific knowledge base to standardize and correct the terminology of the extracted text, providing a highly reliable semantic context for subsequent generation stages.

[0079] 3.1 Multi-level contextualized precise localization. Based on the bounding box coordinates of the chart, text paragraphs beginning with common identifiers such as "Fig.", "Figure," or "Scheme" are searched within a preset geometric range below or above the chart, and these are extracted as the direct description of the chart. The system traverses the full text of the battery literature, using regular expressions to search for all sentences that cite the chart number. These citations, along with one sentence before and after them, are extracted and combined to form an indirect citation context. This part usually contains the analytical conclusions of the experimental data. Each chart obtains a raw context data package containing "Caption text" and "text citation fragments."

[0080] 3.2 Terminology Correction and Standardization Based on Domain Dictionary. The system loads a pre-built battery domain knowledge dictionary, which contains chemical formulas, technical terms, and standard unit symbols for battery materials. It segments the original context data package and calculates the edit distance between word units and dictionary entries. For suspected erroneous words with high similarity, it automatically replaces them using standard spellings from the knowledge base. Regular expressions are used to force correction of subscript and superscript formats, ensuring the professional accuracy of the context information. This results in highly reliable context information after cleaning and correction.

[0081] Step 4: Differentiated prompt engineering construction and high-precision description generation. At the output terminal, the system integrates the classification results from Step 2 with the cleaned context from Stage 3 to dynamically construct structured prompt words for specific chart types. A few-shot learning mechanism is introduced to drive a multimodal large model to generate natural language descriptions that are both visually accurate and possess professional depth.

[0082] 4.1 Dynamic Construction of Differentiated Description Strategies. Based on the routing tags output from Phase Two, the system calls the corresponding prompt word templates to achieve differentiated questioning.

[0083] (1) For data charts: The template focuses on data mining. Example of instructions: Please describe the physical meaning and units of the X-axis and Y-axis in context, analyze the trend of the curve with voltage / time, and clearly point out the key peak data (such as redox peak potential) in the figure and the physical meaning they represent.

[0084] (2) For illustrations: The template focuses on structural analysis. Example of instructions: Please identify the core components in the figure, including the positive electrode, negative electrode and the membrane, describe their spatial relationship including the coating and layered stacking, as well as the microscopic morphological features including spherical particles and cracks, and explain their working mechanism in conjunction with the caption.

[0085] (3) For multi-images: The template focuses on logical connections. Example instruction: This is a set of comparative experimental images. Please describe the contents of sub-image (a) and sub-image (b) respectively, and summarize the performance or structural differences between the two before and after modification in conjunction with the context; where sub-image (a) and sub-image (b) are multiple images from battery literature, and are not the figure numbers of this application.

[0086] 4.2 Few-shot Augmentation and Context Fusion: To standardize the model's output format, the system performs the following fusion operation when constructing the final prompt: (1) Inject typical examples: Retrieve 1-2 high-quality "input-output" pairs that are consistent with the current chart type from the preset "standard description example library". For example, retrieve a standard CV curve description example and insert it into the Prompt as a reference.

[0087] (2) Embedding context information: The semantically enhanced context data package (Clean Context) generated in step three is embedded with... <context> ...< / context> The Prompt is embedded in the form of a label, explicitly telling the model: Please describe based on the visual content of the chart and with reference to the experimental conclusions in the following context information, but do not create illusions.

[0088] 4.3 Multimodal Reasoning and Final Generation: The assembled target prompt is input into the multimodal large model. The target prompt includes charts, instructions, examples and context. The model outputs the final natural language description text. This text not only accurately restores the visual information in the charts, but also correctly cites professional terms and integrates the experimental conclusions in the original document, realizing a deep transformation from visual perception to professional semantics.

[0089] In summary, this invention provides a method and system for generating differentiated descriptions of battery literature images. After acquiring and scanning battery literature, the system divides the images into text, graphic, and background regions according to pixel distribution using a preset algorithm. A unique identifier is assigned to each chart in the graphic region, and a mapping table is constructed based on the chart's document coordinates within the battery literature. Classification prompts are constructed based on preset visual feature boundaries, and a pre-trained multimodal large model is used to semantically classify the target charts in the mapping table to obtain chart types. Processing route labels are assigned to each target chart type. The chart types are parsed to obtain classification metadata, and the target charts are differentiated based on the route labels. Based on the target chart's document coordinates within the battery literature, the system generates differentiated descriptions of the battery literature images. The text of the target chart is extracted from the literature as the direct description text of the target chart. The quotations and related context of the target chart are extracted as the related text. The direct description text and related text are combined to obtain a context data package. After the context data package is processed by a pre-set knowledge dictionary for terminology correction and format standardization, a semantically enhanced context data package is obtained. The corresponding prompt word template is called according to the processing route label to perform differentiated questioning. The description sample with the same chart type as the target chart is retrieved from the pre-set standard description library and inserted into the prompt word template with the semantically enhanced context data package to generate the target prompt word. The chart type and target prompt word are input into a pre-trained multimodal large model to obtain the natural language description text of the target chart.

[0090] This invention also provides an electronic device, which includes: At least one processor; and, A memory that is communicatively connected to at least one processor; wherein, The memory has a computer program that can be executed by at least one processor, such that the at least one processor is able to execute the above-described method for generating differential descriptions of battery literature images.

[0091] This invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the above-described battery document image differential description generation method.

[0092] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for generating differentiated descriptions of battery literature images, characterized in that, The method includes: After acquiring and scanning battery literature, the system divides the region into text, graphic, and background areas according to pixel distribution using a preset algorithm. A unique identifier is assigned to each chart in the graphic area, and a mapping table is constructed based on the document coordinates of the charts in the battery literature. Classification prompts are constructed based on preset visual feature boundaries, and the target charts in the mapping table are semantically classified using a pre-trained multimodal large model to obtain chart types. Processing route labels are assigned to each target chart type. The chart types include data charts, schematic diagrams, multiple charts, tables, and formulas. The chart type is parsed to obtain classification metadata, and the target chart is differentiated according to the routing label. Differentiation of the target chart according to the routing label includes: using a data trend focus mode when processing data charts to identify axis range, curve change trends, peak voltage data characteristics, and current data characteristics; using a principle structure focus mode when processing schematic diagrams to identify microscopic textures, component spatial relationships, and physical component names; using a subgraph relationship focus mode when processing multiple charts to trigger a subgraph segmentation algorithm to identify subgraph index label positions, decompose the combined chart into independent subgraphs, and recursively call the classification process for this stage; and using a dedicated formatting processing mode when processing tables or formulas to generate structured data through an optical character recognition engine and formula code through a formula recognition engine. Based on the document coordinates of the target chart in the battery literature, the caption text of the target chart is extracted from the battery literature as the direct description text of the target chart, the citation statement and related context of the target chart are extracted as related text, the direct description text and the related text are combined to obtain a context data package, and the context data package is processed by a preset knowledge dictionary to perform terminology correction and format standardization to obtain a semantically enhanced context data package. According to the processing route label, the corresponding prompt word template is invoked to perform differentiated questioning. A description sample consistent with the chart type of the target chart is retrieved from the preset standard description library and inserted into the prompt word template with the semantic enhancement context data package to generate the target prompt word. The chart type and the target prompt are input into the pre-trained multimodal large model to obtain the natural language description text of the target chart.

2. The method for generating differentiated descriptions of battery literature images according to claim 1, characterized in that, The process of acquiring and scanning battery literature and then dividing it into text, graphic, and background regions according to pixel distribution using a preset algorithm includes: A page pixel matrix for battery literature is processed using a pixel density clustering-based layout analysis algorithm to identify text line ranges and mark them as text regions. A graphic recognition algorithm based on edge detection and contour extraction is used to process the page pixel matrix to locate image ranges and mark them as graphic regions. A background color consistency analysis algorithm is used to process the page pixel matrix to separate the ranges that match a preset background color threshold and mark them as background regions.

3. The method for generating differentiated descriptions of battery literature images according to claim 1, characterized in that, Classification prompts are constructed based on preset visual feature boundaries, including: The data graph is a visualization that includes coordinate axes, schematic diagrams, and data geometric elements, including cyclic voltammetry graphs, charge-discharge curves, bar charts, and scatter plots; The schematic diagrams described are schematic diagrams illustrating the microstructure of battery materials, battery module structure, or reaction principle, including scanning electron microscope images, transmission electron microscope images, structural disassembly diagrams, and principle schematic diagrams; The multiple images are combined images where a single chart contains at least two independent sub-images and has sub-image index labels; The table is a data table with an explicit row and column grid structure or an aligned text array; The formula is an independent region containing mathematical derivations, chemical reaction equations, or molecular structural formulas.

4. The method for generating differentiated descriptions of battery literature images according to claim 1, characterized in that, The training process for a multimodal large model includes: Obtain a training sample set, which contains multiple samples, each containing a chart from a battery literature document; the multiple samples in the training sample set are labeled with the chart type ground truth and the natural language description ground truth. An initial multimodal large model is obtained, comprising a visual encoder, a text encoder, a multimodal fusion layer, a classifier, and a language decoder. The visual encoder takes samples from the training sample set as input and outputs image view features. The text encoder takes the target prompt words and semantically enhanced context data packets corresponding to the samples as input and outputs text features. The multimodal fusion layer fuses the image visual features and the text features to obtain target features. The target features are then processed by the classifier to output a chart type prediction value, and the target features are processed by the language decoder to output a natural language description prediction value. A first loss function is constructed based on the deviation between the true value and the predicted value of the chart type. A second loss function is constructed based on the deviation between the true value and the predicted value of the natural language description. A joint loss function is constructed using the first and second loss functions. The parameters of the initial multimodal large model are iteratively updated with the goal of minimizing the joint loss function to obtain the multimodal large model.

5. The method for generating differentiated descriptions of battery literature images according to claim 1, characterized in that, The process of performing terminology correction and format standardization on the context data package using a preset knowledge dictionary includes: The preset knowledge dictionary is obtained; the preset knowledge dictionary includes chemical formulas of battery materials, battery-related terminology, and standard unit symbols; The context data packet is segmented to obtain data lexical units, and the edit distance between the lexical units and the preset knowledge dictionary is calculated; when the edit distance is lower than a preset threshold, the words in the context data packet are replaced with standard terms in the preset knowledge dictionary; The subscript and superscript format in the context data packet is standardized using regular expression correction rules.

6. The method for generating differentiated descriptions of battery literature images according to claim 1, characterized in that, The method further includes: storing the chart type, the target prompt word, and the natural language description as a log and saving it to a cloud storage device.

7. A battery document image differential description generation system, characterized in that, The system is used to execute the battery literature image differential description generation method as described in any one of claims 1 to 6, the system comprising: The region division module is used to acquire and scan battery literature and then divide it into text region, graphic region and background region according to pixel distribution using a preset algorithm. It assigns a unique identifier to each chart in the graphic region and constructs a mapping table based on the document coordinates of the chart in the battery literature. The classification processing module is used to construct classification prompts based on preset visual feature boundaries, and to perform semantic classification of the target charts in the mapping table using a pre-trained multimodal large model to obtain the chart type, and to assign processing route labels to each target chart type; the chart types include data charts, schematic diagrams, multiple charts, tables and formulas; the module parses the chart types to obtain classification metadata, and performs differentiated processing on the target charts according to the route labels; The target prompt word generation module is used to extract the caption text of the target chart as the direct description text of the target chart based on the document coordinates of the target chart in the battery literature, extract the citation statements and related context of the target chart as related text, combine the direct description text and the related text to obtain a context data package, and perform terminology correction and format standardization processing on the context data package through a preset knowledge dictionary to obtain a semantically enhanced context data package; call the corresponding prompt word template according to the processing routing label to perform differentiated questioning, retrieve the description sample with the same chart type as the target chart from the preset standard description library and insert it into the prompt word template with the semantically enhanced context data package to generate the target prompt word; The natural language description output module is used to input the chart type and the target prompt word into the pre-trained multimodal large model to obtain the natural language description text of the target chart.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the battery document image differential description generation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the battery literature image differential description generation method as described in any one of claims 1 to 6.

Images