AD-RAG-based vehicle after-sales maintenance document analysis and maintenance knowledge acquisition system and method

Abstract

The application relates to an AD-RAG-based vehicle after-sales maintenance document analysis and maintenance knowledge acquisition system and method, which comprises the following modules: a document analysis module, which analyzes the layout and recognizes the document of after-sales maintenance information to obtain structured data; a knowledge organization module, which establishes an association relationship for text data and structure relationship in the structured data to obtain a maintenance knowledge graph; a knowledge retrieval enhancement module, which analyzes and expands a vehicle fault query request based on an AD-RAG model to obtain an expanded query request, and queries the maintenance knowledge graph based on the expanded query request to obtain document segment information; and a task decomposition and reasoning optimization module, which analyzes and decomposes the expanded query request to obtain at least one fault sub-problem sequence, and performs fault reasoning based on the fault sub-problem sequence and the document segment information through a preset fault reasoning model to obtain a fault maintenance suggestion result. The system can better meet the retrieval requirements and improve the acquisition efficiency of maintenance knowledge.

Description

Technical Field

[0001] This invention relates to the technical field of data retrieval and parsing, and in particular to a system and method for parsing vehicle after-sales maintenance documents and acquiring maintenance knowledge based on AD-RAG. Background Technology

[0002] Adaptive Domain-Specific Retrieval-Augmented Generation (AD-RAG) technology dynamically adjusts retrieval strategies or the generation process to adapt to the complexity and contextual changes of user queries. For example, it can automatically select the search scope or adjust the parameters of the generation model based on the difficulty of the question.

[0003] Currently, vehicle after-sales maintenance typically requires consulting a large amount of technical documentation, including vehicle and component repair manuals, technical service bulletins, fault diagnosis flowcharts, historical repair records, fault logs, etc. This information is often scattered across paper or electronic documents, and retrieving relevant information mainly relies on manual browsing or simple keyword searches.

[0004] Furthermore, vehicle repair documents often contain circuit diagrams, component structure diagrams, tabular data, and technical terms. Traditional text parsing tools struggle to accurately extract useful information from these documents and often overlook the knowledge conveyed by images / charts.

[0005] Therefore, when traditional retrieval systems search for problems during vehicle after-sales maintenance, they often only return results that superficially match the query keywords, which is difficult to meet the retrieval needs in the after-sales maintenance field, resulting in low efficiency in acquiring maintenance knowledge. Summary of the Invention

[0006] This invention provides a vehicle after-sales maintenance document parsing and maintenance knowledge acquisition system and method based on AD-RAG, aiming to meet the retrieval needs of vehicles in the after-sales maintenance field as much as possible, and to make the retrieved maintenance knowledge faster and more accurate.

[0007] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions:

[0008] A vehicle after-sales maintenance document parsing and maintenance knowledge acquisition system based on AD-RAG, including:

[0009] The document parsing module is used to obtain after-sales maintenance document information of the vehicle, and to perform layout analysis and document recognition on the after-sales maintenance document information to obtain structured data. The structured data includes the text data and structural relationships of the after-sales maintenance document information.

[0010] The knowledge organization module is used to establish associations between the text data and structural relationships in the structured data to obtain a maintenance knowledge graph. The maintenance knowledge graph includes nodes that reflect different granularities of information of the text data and edges that connect the nodes. The edges of the nodes are used to reflect the structural relationships between the nodes.

[0011] The knowledge retrieval enhancement module is used to receive a user's vehicle fault query request, parse and expand the vehicle fault query request based on the AD-RAG model to obtain an expanded query request, and perform a query in the maintenance knowledge graph based on the expanded query request to obtain document fragment information, the document fragment information including text data and structural relationships corresponding to the expanded query request;

[0012] The task decomposition and reasoning optimization module is used to parse and decompose the extended query request to obtain at least one sequence of fault sub-problems, and to perform fault reasoning based on the sequence of fault sub-problems and the document fragment information through a preset fault reasoning model to obtain fault repair suggestion results.

[0013] Accordingly, this application also provides a method for parsing vehicle after-sales maintenance documents and acquiring maintenance knowledge based on AD-RAG, including:

[0014] The system acquires vehicle after-sales maintenance documentation information, performs layout analysis and document recognition on the after-sales maintenance documentation information to obtain structured data, which includes the text data and structural relationships of the after-sales maintenance documentation information.

[0015] A relationship is established between the text data and structural relationships in the structured data to obtain a maintenance knowledge graph. The maintenance knowledge graph includes nodes that reflect different granularities of information of the text data and edges that connect the nodes. The edges of the nodes are used to reflect the structural relationships between the nodes.

[0016] The system receives a user's vehicle fault query request, parses and expands the vehicle fault query request based on the AD-RAG model to obtain an expanded query request, and performs a query on the maintenance knowledge graph based on the expanded query request to obtain document fragment information, which includes text data and structural relationships corresponding to the expanded query request.

[0017] The extended query request is parsed and decomposed to obtain at least one sequence of fault sub-problems. Based on the sequence of fault sub-problems and the document fragment information, fault reasoning is performed using a preset fault reasoning model to obtain fault repair suggestion results.

[0018] The beneficial effects of this invention are as follows:

[0019] 1. This invention can convert vehicle after-sales documents into digital data through a document parsing module. During the data retrieval process, relevant personnel do not need to flip through paper documents, which can greatly improve the speed of data retrieval.

[0020] 2. The document parsing module can also perform layout analysis and recognition on the structure and circuit diagrams in maintenance documents, enabling relevant personnel to conduct more comprehensive information searches by combining relevant diagrams and tables when viewing the documents.

[0021] 3. By combining the knowledge organization module, the text data and structure in vehicle after-sales maintenance documents can be correlated. When a fault query request is received, the knowledge retrieval enhancement module can quickly find relevant documents in the maintenance knowledge graph through the correlation established by the knowledge organization module, thereby improving the efficiency of information retrieval. At the same time, through the analysis and expansion of the problem, it can better meet the retrieval needs in the maintenance field.

[0022] 4. Through the task decomposition and reasoning optimization module, query requests can be parsed and decomposed, and fault reasoning can be performed through the fault reasoning model to obtain more accurate fault repair suggestions, thereby improving the efficiency of acquiring maintenance knowledge. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A schematic diagram of the structure of a vehicle after-sales maintenance document parsing and maintenance knowledge acquisition system based on AD-RAG, provided for the application embodiment;

[0025] Figure 2 This is a flowchart illustrating a method for parsing vehicle after-sales maintenance documents and acquiring maintenance knowledge based on AD-RAG, provided for the application embodiment. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Figure 1This is a schematic diagram illustrating the structure of a vehicle after-sales maintenance document parsing and maintenance knowledge acquisition system based on AD-RAG, provided as an embodiment of this application. Figure 1 As shown, this AD-RAG-based vehicle after-sales maintenance document parsing and maintenance knowledge acquisition system includes:

[0028] Document parsing module 1 is used to obtain after-sales maintenance document information for vehicles, and to perform layout analysis and document recognition on the after-sales maintenance document information to obtain structured data. The structured data includes the text data and structural relationships of the after-sales maintenance document information.

[0029] There are various ways to obtain vehicle after-sales maintenance documentation. This can include scanning, obtaining photos, direct electronic document transmission, or directly accessing data sources, such as retrieving maintenance documents from the automaker via a network connection. After-sales maintenance documentation includes text, images, circuit diagrams, tables, etc., and may include vehicle and component maintenance manuals, technical service bulletins, fault diagnosis flowcharts, fault records, and historical repair records.

[0030] The document parsing module 1 performs layout analysis and document recognition on the after-sales maintenance documents. This involves analyzing the layout of the original documents, extracting the document's chapter structure, paragraph content, chart information, etc., to form structured data. In this process, relevant document content can be obtained while preserving the hierarchical relationships within the document.

[0031] Furthermore, in some embodiments, the document parsing module 1 includes:

[0032] An optical character recognition unit is used to recognize after-sales maintenance document information and digitize the recognized after-sales maintenance document information to obtain the first text data.

[0033] The layout analysis unit is used to perform layout analysis on after-sales maintenance document information to obtain the structural relationship of the document content.

[0034] The visual language generation unit is used to extract display images of after-sales maintenance document information and annotate the display images based on a pre-trained visual language model to obtain second text data.

[0035] The text data includes first text data and second text data.

[0036] In the optical character recognition unit, dynamic error correction can be performed based on the document's context. Let the candidate character sequence of the input document image be C = {c1, c2, ..., c...}. n}, ci The candidate set for the i-th character (e.g., character candidates generated through image segmentation, including multi-character hypotheses, such as "turbocharger" can be segmented into "turbo", "charger", or a single complete word). The position coordinate matrix is ​​P = {p1, p2, ..., p...} n}, p i =(x i ,y i ,w i ,h i ): The coordinate information of the i-th candidate character, including the coordinates of the top-left corner (x, y, y). i ,y i Width w i Height h i This is used to locate the physical position of characters on the page. A context vector H = Transformer(C,P), where H = {h1, h2, ..., h...}, is generated by the Transformer encoder. n}, each h i It is a feature vector that integrates contextual semantics and location information, combined with a conditional random field (CRF) for sequence error correction:

[0037]

[0038] Where T1 is the first text data (character sequence), containing location-aware character embeddings LocalFeat(p i R represents the structural relationships (column boundaries, cross-page associations, and other spatial constraints) output by the layout analysis unit, and α represents... i ,β i ,γ i,i+1 For dynamic weights, the weights are adaptively adjusted based on the layout type (e.g., table, plain text). Attn(H i LocalFeat(p) is an attention mechanism based on structural relationships R, which calculates the association between the i-th character and structural features such as column boundaries and cross-page associations. For example, when a character is located at a column boundary, it enhances attention to characters in adjacent columns to avoid cross-column misidentification (such as correctly associating "fault" in the left column with "code" in the right column). i `T1[i]` extracts local features of character positions (such as distance relative to column boundaries and inline alignment) to determine whether a character belongs to a table cell or a plain text area, and dynamically adjusts the error correction strategy (characters within a table strictly adhere to a fixed format, such as the fault code "P0171" which must contain both letters and numbers). `CRFTransition(T1[i],T1[i+1])` models the transition probabilities of character sequences, such as the probability of "P" being followed by a number being much higher than that followed by a letter (utilizing the prior syntax of fault codes in maintenance documents), correcting OCR's misjudgments of similar characters (such as distinguishing between "0" and "O").

[0039] In this process, the layout structure relationship R is introduced as a guiding signal for the Transformer attention mechanism to solve the problem of character misidentification across columns in multi-column layouts (such as character confusion between table columns), and the position coordinates p are used to solve the problem. i By integrating with structural relation R, we can achieve sentence breakage repair for continuous text spanning multiple pages (such as punctuation completion when maintenance steps span multiple pages).

[0040] In the layout analysis unit, the document structure can be parsed and identified through semantic association of hierarchical graph structure, dividing the document layout into a set of regions A = {a1, a2, ..., a...}. m}, a i This represents the i-th layout area (such as a table cell, a piece of text, or an illustration), where each area contains the first text data fragment generated by OCR. Image features V generated by visual language a Construct a hierarchical graph structure Where: edge weight E i,j =σ(GNN) spatial (P i ,P j )+GNN semantic (T1 i T1 j Logical order vector L = Transformer order (A, E)⊙PositionalEncoding(P), the structural relationship output is: in: V represents the first text data in region i (such as an error code within a cell). a The image semantic vector (such as part schematic features) output by the visual language generation unit is used to generate the image semantic vector. This indicates the image and text feature fusion operation.

[0041] Hierarchical graph structure In the diagram, node A represents a page area, characterized by: (Text and visual feature concatenation). Edge E represents edge weight E. i,j The degree of association between regions i and j is represented by spatial relationships (such as distance and overlap) and semantic relationships (such as text similarity). The logical order vector L = {l1, l2, ..., l...} m} represents each l i Indicates the position of region i in the logical reading order (e.g., "column 1, line 3" or "page 2, line 5 of a cross-page table").

[0042] The structural relationship output R consists of three parts: ColumnLayout (A): column layout parameters (such as column width, column spacing, and cross-column area markers). CrossPageMerge (L): row merging rules for cross-page tables (e.g., row 5 of page 1 and row 1 of page 2 belong to the same row). The reading order of mixed text and graphics (e.g., "text description first, then circuit diagram on the right" or "icon first, then description below").

[0043] Among them, GNN spatial (P i ,P j ) represents a spatial graph neural network, where the input regions are the position coordinates P of i and j. i ,P j It outputs spatial correlation (e.g., adjacent areas have high weights, cross-page areas have low weights) to identify column boundaries and table page continuity. This represents a semantic graph neural network, with OCR text input to regions i and j. Calculate text similarity (e.g., the semantic association between "fault code" and "P0171") to determine whether regions belong to the same logical module (such as the description of the same fault). PositionalEncoding(P) encodes the region's position coordinates into an order vector, ensuring that the text within a column is sorted "from top to bottom and from left to right," correcting order errors caused by column segmentation in OCR (e.g., incorrect splicing of text between left and right columns).

[0044] In this process, a cross-modal graphical neural network (GNN) is constructed, which simultaneously takes OCR text T1, position coordinates P and visual semantics V as input to resolve logical order ambiguities in mixed text and image scenarios (such as the correspondence between circuit diagram numbers and text descriptions).

[0045] By explicitly modeling column layout and cross-page table merging using a hierarchical graph structure, the order information in L is used to correct text order errors caused by column segmentation in OCR (such as text interleaving between left and right columns).

[0046] Introducing visual semantic vectors V enhances semantic correlation calculation. For example, it can determine whether a text area belongs to the same part description by using the features of the part diagram, thereby improving the structural parsing accuracy of complex layouts.

[0047] In the visual language generation unit, let the input image feature be I, and the pre-trained visual language model outputs the initial annotation. Introducing a vehicle-specific knowledge base (e.g., fault light icon-code pair), generating second text data through comparative learning and cross-validation:

[0048] in:

[0049] The cross-validation function CrossVerify is defined as follows:

[0050]

[0051] T1 represents the first text data generated by OCR, R represents the structural relationships of the layout analysis (such as regional proximity), and d(I,V) k The distance is the cosine distance between the image features and the knowledge base features. (Vehicle-specific knowledge base) V k The k-th image feature in the knowledge base (e.g., the standardized "engine malfunction light" icon feature). T k : The corresponding standard text description (such as "P0171 System Too Thin" "MIL Light"). To compare the loss function, the distance between the input image feature I and similar features in the knowledge base (such as the "engine fault light" family of icons) is forced to be smaller than that of dissimilar features (such as "ABS light"), thereby improving the labeling accuracy of domain-specific images (such as distinguishing between different codes for "always on fault light" and "flashing"). Cross-validation mechanism: Structural consistency check: Utilizing proximity constraints in the layout structure relationship R (e.g., the text "turbocharger" and the illustration must be adjacent in the layout), determine the consistency between the OCR text T1 and the visual annotation. Do they belong to the same region? Knowledge base matching: If the structures are inconsistent, search the knowledge base. V is the closest to image feature I k Get the corresponding standard text T k (For example, matching the typical number of blades in a "turbocharger" using schematic structural features). d(I,V k ) is the cosine distance between the image features and the knowledge base features, used to quantify similarity, and a threshold is set to filter valid matches (e.g., if the distance is less than 0.2, it is considered to be the same icon).

[0052] In this process, a knowledge base contrastive loss is introduced to force alignment of specific mapping relationships such as "fault light icon-code" and "part diagram-name" (e.g., binding the engine fault light image with the "P0171" code) to solve the labeling bias of the general model for vehicle domain terms.

[0053] Based on the cross-validation mechanism of layout structure relationship R, when OCR recognizes "turbocharger", it locates the adjacent schematic diagram area through R and uses the knowledge base to verify whether the schematic diagram structure conforms to the characteristics of the component (such as the number of blades and the layout of connecting pipes), thus correcting the OCR's misidentification of similar terms (such as "turbo compressor").

[0054] By using bidirectional constraints of T1 and T2 (text annotation guides image region localization, and image features verify text semantics), a closed-loop verification of "OCR-layout-visual" is formed, improving the parsing robustness of complex mixed text and image documents.

[0055] Finally, text data T is generated by fusing the first text data T1 and the second text data T2 under the constraint of structural relation R: in: Representing feature splicing or attention-weighted fusion, R provides structural information such as column order, cross-page merging, and image-text correspondence as positional encoding and attention masks for the fusion process.

[0056] Through the above interaction process, the character positions P and the initial text T1 output by OCR serve as node features of the layout analysis graph model. The structural relationship R of the layout analysis guides the sequence correction of OCR (such as sentence segmentation correction at column boundaries). The region division A and logical order L of the layout analysis provide the location of the region of interest (ROI) for visual language generation (such as automatic cropping of circuit diagrams and fault light icons). The semantic vector V generated by the visual language enhances the semantic relevance calculation of the layout analysis (such as the semantic similarity of text and image regions). The second text data T2 generated by visual analysis (such as "P0171") and the first text data T1 of OCR (such as "system too sparse") are processed through a knowledge base. Establish a mapping and combine it with the proximity constraint in the layout structure relationship R to achieve the triangulation of "term-icon-description" (such as ensuring that the fault code description and the corresponding icon are adjacent in the layout).

[0057] Ultimately, through the three units mentioned above, a deep coupling of "recognition-structure-semantics" can be formed. The optical character recognition (OCR) unit, based on traditional text recognition, injects layout structure R and visual semantics V to perform contextual error correction and generates the first text data T1 with positional information.

[0058] The layout analysis unit constructs a cross-modal graph structure, integrates the text features of T1, the image features of visual V, and the position P, and outputs a structural relationship R that includes columns, order, and cross-page relationships.

[0059] Visual language unit utilizes knowledge base Given the structural relationship R, we perform domain augmentation and cross-validation on the image annotation T2 to ensure semantic and structural consistency with T1. Finally, through a fusion model, under the constraints of R, we integrate T1 and T2 to generate text data that conforms to human reading logic, has a complete structure, and is semantically accurate, thus solving the problems of "isolated recognition, structural fragmentation, and disconnect between text and images" in traditional document parsing.

[0060] Furthermore, in some embodiments, document recognition is performed on the after-sales service documentation information, and the document content of the recognized after-sales service documentation information is digitized to obtain first text data, including:

[0061] After-sales maintenance document information is identified, and the identified after-sales maintenance document information is fed into a pre-trained large language segmentation model for document segmentation to obtain at least one document block. Each document block includes a complete semantic unit and a document topic corresponding to the semantic unit.

[0062] The document content of the document block is converted into data to obtain the first text data.

[0063] Suppose that the after-sales service document information is D, and it is recognized by optical character recognition (OCR) technology to obtain the initial text T0. T0 is then fed into a pre-trained large language segmentation model M. chunk Perform document chunking. The output of the large language chunking model can be represented as a set of document chunks. Each document block B i Contains a complete semantic unit S i and corresponding document themes

[0064] To achieve more accurate segmentation, considering the correlation and hierarchical structure of different types of information (such as fault descriptions, repair procedures, and parts information) in vehicle repair documents, a context-aware segmentation weight function w is introduced. context This function combines the position coordinate matrix P in the optical character recognition unit and the structural relationship R of the layout analysis unit:

[0065]

[0066] Here, α and β are adjustable weight parameters, and Rel(T0[i],R) represents the relevance score of the i-th character in text T0 to the structural relationship R, which can be obtained by calculating the degree of matching between the character position and spatial constraints such as column boundaries and cross-page associations.

[0067] The document segmentation process can be represented as: B i =M chunk (T0,w context (T0,P,R)).

[0068] Each document block B i The document content is digitally transformed to obtain the first text data T. 1i Considering the large amount of technical terminology and structured information in vehicle repair documentation, such as fault codes and part numbers, a domain-knowledge-enhanced data transformation function f is introduced. transform .

[0069] Let the vehicle-specific knowledge base be Document Block B i The semantic unit in S i The data conversion formula is: Among them, f transform Combined with knowledge base The structural relationship R between information and layout analysis units in the semantic unit S i Processing is required. Specifically, f transform This can be achieved through the following steps:

[0070] Term mapping: based on knowledge base semantic unit S i Technical terms are mapped to standard terminology. For example, "engine malfunction indicator lamp" is mapped to the term "P0171 system too lean".

[0071] Structural adjustment: Adjust semantic units S according to structural relations R. i The order and format are adjusted to better suit the requirements of structured data. For example, maintenance steps spanning multiple pages can be combined into a single, complete sequence. Then:

[0072] Here, ° represents the combination of functions.

[0073] Finally, the final first text data T1 = {T 11 ,T 12 ,…,T 1n The document is composed of the digitized transformation results of all document blocks. In this process, the entire workflow is closely linked to the Optical Character Recognition (OCR) unit, the layout analysis unit, and the visual language generation unit: the OCR unit provides the initial text T0 and the position coordinate matrix P, providing the basic data for document segmentation and digitization. The structural relationship R of the layout analysis unit is used for context-aware segmentation weight calculation and structural adjustment during digitization, ensuring the accuracy of segmentation and data transformation. The visual language generation unit has a knowledge base. Terminology mapping used in data transformation improves the semantic accuracy of data.

[0074] In the above process, the large language chunking model M chunk Document segmentation can be achieved in the following ways:

[0075] The input to the large language chunking model not only includes the initial text T0 generated by OCR, but also integrates two key auxiliary information:

[0076] The position coordinate matrix P of the optical character recognition unit records the spatial position information of each character / text block in the document, such as the two-dimensional coordinates (x, y), column affiliation (e.g., left column / right column), and page crossing identifier (e.g., whether it is located at the beginning / end of the page), which is used to characterize the physical layout features of the document.

[0077] The structural relationship R of the layout analysis unit: Through the column adaptive algorithm and logical reading order model, semantic structural features such as the hierarchical structure of the document (such as title-subheading-body text), column correlation (such as row and column mapping of multi-column tables), and cross-page continuity (such as the page connection relationship of maintenance steps) are extracted.

[0078] Formula association: Input = {T0, P, R} where T0 is the original carrier of the first text data, P and R are the spatial representation and logical representation of the structural relationship, respectively, and the three together constitute the input space of the block model.

[0079] The model uses a context-aware block weight function w context The "segmentation cost" of each potential block boundary in the text is dynamically calculated to ensure that the segmentation results conform to human reading logic. This is done in two steps:

[0080] 1. Semantic Relevance Calculation

[0081] Deep semantic encoding based on large language models (such as BERT) extracts the context embedding vector Emb(T0[i]) for each position i in the text sequence T0, and uses topic modeling (such as LDA) to determine whether the current position belongs to the semantic unit boundary (such as paragraph end, topic change). SemanticScore(i) = Softmax(MLP(Emb(T0[ik:i+k]))) where k is the size of the context window, and a higher SemanticScore(i) indicates that this position is more likely to be a semantic block boundary.

[0082] 2. Structural constraint fusion

[0083] Introducing spatial location weights α·P i The logical structure weight β·Rel(T0[i],R) is used to calculate the matching degree between text and structure. The former reflects the physical position of the text block in the layout (such as whether it is in the same column or at the page transition), while the latter calculates the matching degree between text and structure based on the structural relationships obtained from layout analysis (such as column affiliation and table page transition marks). context (i)=γ·SemanticScore(i)+α·P i +β·Rel(T0[i],R) where γ, α, and β are learnable parameters. By training, the weights of the three are optimized so that the block segmentation results simultaneously satisfy semantic integrity (e.g., "fault code description" is not segmented) and structural continuity (e.g., cross-page tables are merged into the same document block).

[0084] In the process described above, traditional segmentation relies solely on textual semantics (such as punctuation and paragraph marks). Here, however, page layout information is introduced through P and R, resolving segmentation errors caused by layouts such as "multi-column mixed layout" and "cross-page tables" in vehicle repair documents. For example, when OCR recognizes "Repair Step 1" at the end of page 1 and "Repair Step 2" at the beginning of page 2, the cross-page continuity marker in R will reduce the segmentation weight at this point, forcing them to be merged into the same document block.

[0085] Each output document block Two core conditions must be met:

[0086] Semantic unit integrity S i It includes independent semantic functions, such as "fault diagnosis process", "parts replacement steps" and "parameter table interpretation", avoiding separating "fault code definition" and "corresponding repair solution" into different blocks.

[0087] Theme Consistency By using the topic classification head of the large language model, a unique topic label (such as "engine failure" or "circuit system maintenance") is assigned to each block, and the label space is aligned with the vehicle maintenance knowledge base (such as fault code library and parts manual).

[0088] Formulaic expression:

[0089] B i =argmin B (λ1·SemanticDiscontinuity(B)+λ2·StructuralDiscontinuity(B)) where SemanticDiscontinuity(B) is the semantic discontinuity within the block (calculated by the cosine distance of the semantic embedding), StructuralDiscontinuity(B) is the structural discontinuity within the block (such as the penalty for not merging across columns or pages), and λ1 and λ2 are hyperparameters that balance semantics and structure.

[0090] Furthermore, the large model can be associated with three units, including the optical character recognition unit: the position coordinate matrix P provided by OCR is the basis for the block model to perceive the physical layout of the document. For example, through the column coordinates in P, the model can identify the multi-column area where the "part parameter table" is located, avoiding the division of the same table content into different blocks.

[0091] Relationship with layout analysis units: The structural relationships R obtained from layout analysis (such as column boundaries and cross-page associations) are directly input into the weight function of the block model to ensure that the block results conform to the logical structure of the document (such as naturally dividing different types of content according to column boundaries and merging continuous tables according to cross-page markers).

[0092] Collaboration with visual language generation units: Truncate document topics Provide semantic anchors for the visual language generation unit. For example, when the block topic is "fault indicator light icon", the visual unit can specifically retrieve icon images from the vehicle-specific knowledge base to achieve accurate association between "icon and text" (such as automatic matching of "engine fault light" image with corresponding fault code text).

[0093] The technical solutions provided in the above embodiments have the following technical advantages:

[0094] 1. By using an optical character recognition unit and introducing page layout relationships R (column boundaries, page cross-page associations) as attention-guided signals, the problem of character cross-column confusion in multi-column layouts such as part parameter tables and fault code tables (e.g., misjudging "P0171" and similar codes) is solved. Combined with the terminology and grammar rules specific to CRF modeling maintenance documents (e.g., fault code format, component name spelling), the recognition error of professional terms such as "turbocharger" is reduced. This improves the accuracy of professional terminology recognition. The position coordinates p are used... i The integration with structural relationship R automatically completes the punctuation marks for cross-page maintenance steps (such as missing periods and conjunctions in cross-page tables), ensuring the coherence of the process description.

[0095] 2. By incorporating an optical character recognition (OCR) unit and introducing a context-aware chunking weight function, the spatial structure and contextual information of the document are considered, enabling more accurate identification of semantic units and topics within the document. For example, in a vehicle repair manual, different columns contain different types of information. By combining positional coordinates and structural relationships, information belonging to the same topic can be avoided from being segmented into different document blocks, thus improving the accuracy of chunking.

[0096] 3. The layout analysis unit integrates OCR text T1, position coordinates P, and visual semantics V (such as circuit diagram features) through a cross-modal graph neural network to accurately establish the correspondence between circuit diagram numbers and text descriptions, avoiding confusion in the parsing order caused by the proximity of images and text. Furthermore, the hierarchical graph structure explicitly models column layouts (such as left and right column technical parameters) and merges cross-page tables (such as maintenance procedure tables spanning multiple pages), using a logical order vector L to correct text interleaving errors caused by column segmentation (such as correct alignment of left column part names with right column parameters), ensuring that the parsing results conform to human reading habits.

[0097] 4. The visual language generation unit utilizes a vehicle-specific knowledge base. By employing comparative learning (such as binding brand logos, fault indicator icons, and codes), automatic semantic alignment is achieved between "engine fault light image" and "P0171 system too sparse," resolving the annotation bias specific to areas like "ABS fault light" and "coolant temperature alarm icon" in general models, thereby improving the accuracy of association. Furthermore, based on the cross-validation mechanism of layout structure relationship R, when OCR recognizes "turbocharger," it automatically locates the adjacent schematic area and verifies structural features such as the number of blades and connecting pipes through a knowledge base, eliminating misjudgments of similar terms like "turbo compressor," improving the semantic consistency between component names and schematic diagrams, and reducing misrecognition in mixed text and image scenarios.

[0098] 4. By combining the above three units, cross-page tables can be automatically merged, and column content can be correctly grouped, avoiding table breaks caused by page cutting (such as continuous row information when maintenance steps cross pages), and ensuring the integrity of the structure as much as possible. Fault codes, icons, and text descriptions are verified through knowledge base and structural proximity (such as the three being related in position and semantically matched on the page), eliminating problems such as "disconnect between images and text" and "mismatch between terminology and icons" in traditional parsing, and ensuring semantic consistency as much as possible. At the same time, mixed text and image content is output in the order of human reading (such as images first, then text descriptions, and table parameters aligned by column), which greatly reduces the understanding cost for maintenance personnel and improves document usage efficiency, making document reading conform to the reading logic of ordinary people.

[0099] Knowledge organization module 2 is used to establish associations between text data and structural relationships in structured data to obtain a maintenance knowledge graph. The maintenance knowledge graph includes nodes that reflect different granularities of information in the text data and edges that connect the nodes. The edges of the nodes are used to reflect the structural relationships between the nodes.

[0100] The maintenance knowledge graph includes:

[0101] Document layer nodes include complete data on after-sales maintenance documentation;

[0102] Chapter-level nodes include chapter structure data for after-sales maintenance documentation information;

[0103] Paragraph layer nodes include paragraph data for after-sales maintenance documentation information;

[0104] Entity relationship layer nodes include vehicle part entity data;

[0105] The structural relationships between corresponding nodes in the maintenance knowledge graph are established by connecting document-level nodes, chapter-level nodes, paragraph-level nodes, and entity-relationship-level nodes through edges.

[0106] The document layer nodes include a complete dataset of after-sales document information, such as a complete document or data source, and different document layer nodes are connected by edges; the chapter layer nodes include the chapter structure data of after-sales repair document information, that is, the hierarchical chapter structure within each document; the paragraph layer nodes include the paragraph data of after-sales repair document information, that is, specific knowledge paragraphs or entries; the entity relationship node layer includes vehicle part entity data, that is, part entities extracted from the repair corpus and relationship nodes.

[0107] For example, taking the construction of a knowledge graph for vehicle engine fault repair as an example, in the scenario of vehicle engine fault diagnosis, the repair manual contains multi-dimensional information (such as system schematic diagram, fault code table, disassembly and assembly steps). The document content needs to be organized according to the hierarchy of "document-chapter-paragraph-entity" and a semantic association of "fault code-part-repair steps" needs to be established.

[0108] First, embedding at the document layer node: let the optical character recognition result of the input document D be T1, the layout analysis structural relationship be R (including column and page span information), and the system schematic feature for visual language generation be V. sys Define document layer node embedding:

[0109] Where: PosEnc(R) page ) is the positional encoding of the cross-page order (based on page layout analysis of cross-page merging relationships R) page ), GraphPooling(V sys Pooling is performed on the graphical structural features (such as component connection relationships) of the engine system schematic diagram.

[0110] Next, embed the column layout R based on the page layout analysis at the chapter level node. col Divide the document into chapter C i Each chapter contains a column of text (T). 1i and the corresponding fault light icon feature V icon Embedded formula: Among them: GNN column For the column-aware graph neural network, the input includes the text within the column, the coordinates of the column boundaries, and icon features. AttnMerge fuses the global embedding of the document layer and the local features of the chapter through an attention mechanism.

[0111] Then, embedding document block B, output by the large language chunking model, at the paragraph level node. j (including semantic unit S) j and theme ), combined with OCR position coordinates P j and visually generated part schematic features V part :

[0112] Where: PositionGate(P j This is a position-aware gating unit that determines whether a paragraph spans multiple pages based on its coordinates. KnowledgeEmbed uses the vehicle knowledge base... Extracting topic-related prior knowledge (such as standard failure modes of "engine fuel system"), ⊙ represents the Hadamard product, which is element-wise multiplication. This indicates vector concatenation or feature fusion.

[0113] Finally, embedding the part entity E in the entity relationship layer node defines the part entity E. part Fault code entity E code Repair steps Entity E step Embedded formulas integrate visual features with textual semantics: Where: V part T1 outputs a visual language model of the part schematic diagram (such as the 3D features of a turbocharger), T2 is the annotation text generated by the visual language (such as "P0171 System Too Sparse"), and ContrastiveEncoder forces the alignment of the "part image-fault code-text description" triple through contrastive learning.

[0114] After the nodes of each layer are embedded, it is necessary to establish relationships between different layers. In the document layer and the chapter layer, the edge weights are... This reflects the structural importance of chapters within the document, combined with the proportion of column area and semantic similarity: in The area of ​​each column is calculated using the position coordinates from layout analysis. λ1 and λ2 represent adjustable weight parameters, and their sum is 1. TotalArea(D) represents the total area of ​​the entire document (the sum of the areas of all pages), used to normalize the proportion of column area. Represents the document layer embedding vector E doc Embedded vector of the i-th chapter layer The cosine similarity is used to measure the semantic consistency between the two.

[0115] At the chapter and paragraph levels, the logical reading order vector L and paragraph position coordinates P are based on page layout analysis. j Define edge weights: Among them: GNN order Modeling the reading order between paragraphs (such as the sequence of maintenance steps), TF-IDF measures the paragraph topic. Related to Chapter C iThe correlation (such as the "Injector Cleaning Steps" paragraph in the "Fuel Injection System" chapter), σ represents the activation function (such as Sigmoid), which converts the weighted sum into probability values ​​in the range [0,1] as edge weights.

[0116] In the paragraph and entity layers, a vehicle-specific knowledge base is used. Establish a mapping between paragraph terms and entities, with edge weights as follows: Where TripleLoss is the "text-entity-knowledge base" triple loss, when the terms in the paragraph (such as "turbocharger") match the entity features (E... part When the definition of the knowledge base is consistent with the definition of the knowledge base, the loss value is minimized and the edge weight is maximized.

[0117] The technical solutions provided in the above embodiments have the following technical advantages:

[0118] 1. Traditional solutions typically model documents as a flat structure (e.g., entity-relationship tuples only), lacking hierarchical modeling of "document-chapter-paragraph-entity," and failing to distinguish dependencies between different granularities of information. For example, the inclusion relationship between chapter topics and paragraph semantics is weakened. This application defines a four-layer node system: document layer, chapter layer, paragraph layer, and entity-relationship layer, and uses cross-layer edge weight formulas (e.g.) Explicitly model hierarchical relationships. For example, embedding chapter-level nodes. Fusion document-level global features (E doc It retains the structural location of chapters in the document and supports multi-granularity reasoning from macro (document topic) to micro (entity relationship).

[0119] 2. In traditional solutions, edge calculations between nodes are based solely on textual semantic similarity (such as TF-IDF or BERT embedding matching), without considering the mapping between the document's physical and logical structures. Paragraphs and entities within the same column are incorrectly separated due to mismatched text keywords, or irrelevant content across columns is incorrectly associated. This application calculates edge weights through structure-relationship-driven methods: for example, document-level-chapter-level edge weights. By combining column area ratio (physical structure) and semantic similarity (logical structure), the importance of chapter nodes in the knowledge graph is ensured to be consistent with the actual document layout; paragraph-entity layer edge weights (W) para-entity This approach uses Triple Loss to force alignment between terms, entities, and knowledge base definitions. By establishing a dual constraint of "structure + semantics," the association between the "turbocharger" paragraph and its corresponding part entity is not only based on text matching but also on the features of adjacent diagrams (V...). part ) and knowledge base definition validation to avoid incorrect associations caused by ambiguity in terminology.

[0120] 3. Traditional solutions rely on general knowledge graph models, lacking vehicle-specific entity types (such as fault codes and part diagrams) and relationships (such as "fault code-part-repair procedure" triples). For example, they cannot accurately identify the specific association between "P0171" and "engine fuel system too lean," or they confuse the unique terminology of different vehicle brands. This application integrates a vehicle-specific knowledge base. And contrastive learning mechanisms: for example, entity relation layer node embedding (E entity By aligning the "part image-fault code-text description" triples through comparative learning, paragraph terminology mapping utilizes knowledge base definitions (e.g., "engine fault light" directly associates with "P0171"). By constructing a domain-specific entity relationship network, such as a closed-loop knowledge chain of "symptom (insufficient power)-fault code (P0171)-part (injector)-repair procedure," inference can accurately locate the corresponding chapters and paragraphs in the manual, avoiding the domain generalization error of general models.

[0121] 4. Traditional solutions only support shallow keyword-based retrieval and cannot handle cross-modal reasoning (such as text-image linkage) or structural constraint reasoning (such as cross-page step completeness). For example, when a repair technician searches for "turbocharger fault," traditional solutions cannot simultaneously display the schematic diagram of the part in the manual and the disassembly and assembly steps across pages, resulting in information fragmentation. In this application, deep reasoning is supported through multimodal hierarchical embedding and structural relationship constraints: for example, during fault diagnosis, the knowledge graph can be used through document layer nodes (E... doc Verify the structural consistency of the chapters to which the maintenance procedures belong, through visual features (V). part This involves associating component diagrams with the system to achieve multi-dimensional information fusion of "text description, structural location, and visual image." It upgrades traditional "keyword matching" to joint reasoning of "structure + semantics + vision." For example, when recommending repair steps, it automatically merges cross-page content and marks the corresponding component diagram locations, improving diagnostic efficiency.

[0122] The knowledge retrieval enhancement module 3 is used to receive users' vehicle fault query requests, parse and expand the vehicle fault query requests based on the AD-RAG model to obtain expanded query requests, and perform queries in the maintenance knowledge graph based on the expanded query requests to obtain document fragment information, which includes text data and structural relationships corresponding to the expanded query requests.

[0123] When a vehicle fault query request is received from a user or maintenance personnel, the request includes the fault problem of the vehicle. Then, based on the AD-RAG model, the vehicle fault query request is parsed and expanded. For example, if the received fault problem is "battery dead", after parsing, it can be determined as "battery power insufficient", and then it can be further expanded, such as "battery voltage low" or "12V battery voltage low", thus obtaining an expanded query request. Then, based on the expanded query request, the maintenance knowledge graph can be searched for document data that matches the expanded query request, thereby obtaining the document fragment information of the corresponding fault. The obtained document fragment information includes text data and the structural relationship of the text data.

[0124] Furthermore, in some embodiments, the knowledge retrieval enhancement module 3 is also used to obtain vehicle-related professional names and corresponding colloquial names, and to establish a tag comparison library based on the professional names and colloquial names, and to establish an association between the tag comparison library and the maintenance knowledge graph.

[0125] The vehicle fault query request is parsed and expanded based on the AD-RAG model to obtain an expanded query request, including:

[0126] The method describes parsing vehicle fault query requests based on the AD-RAG model to obtain fault keywords;

[0127] Expand the fault keywords to obtain fault synonyms and / or fault near-synonyms;

[0128] Based on fault synonyms and / or fault near-synonyms, an expanded query request is obtained;

[0129] Based on the extended query request, a query is performed in the maintenance knowledge graph to obtain document fragment information, including:

[0130] Match fault synonyms and / or fault near-synonyms with the tag lookup library to obtain the corresponding professional names, which will be used as the query names;

[0131] The query is performed in the maintenance knowledge graph based on the query name to obtain document fragment information.

[0132] The following example illustrates how, in a vehicle repair scenario, a user describes a fault in colloquial language (e.g., "brakes are soft"). The system needs to use the AD-RAG model to parse the query intent, combine it with a tag lookup library to complete the mapping from "colloquial words to technical terms" (e.g., "brakes are soft" → "air ingress into the braking system"), and finally retrieve document fragments containing structural relationships (e.g., the cross-page paragraph of the brake fluid bleed procedure, the location of the brake system diagram) from the repair knowledge graph.

[0133] First, a relationship model is established between the tag reference library and the knowledge graph, defining the tag reference library as a bipartite graph. in: For a set of spoken name nodes (such as "accelerator" and "brake"), For a set of professional name nodes (such as "throttle body" and "brake master cylinder"), edge E map The mapping relationship is represented by weights determined jointly by domain semantic similarity (DomSim) and knowledge graph structural relevance (StrAssoc): in:

[0134] DomSim is based on a vehicle knowledge base. The terminology definition calculation (e.g., the standard definition of "throttle" includes the keyword "throttle valve"), The structural importance of terminology nodes in the knowledge graph is calculated using graph neural networks (node ​​degree, betweenness centrality, etc.). α and β represent learnable weight parameters used to balance the importance of domain semantic similarity (DomSim) and knowledge graph structural relevance (StrAssoc).

[0135] Secondly, cross-modal tag embedding is performed to generate spoken name c col and professional name c tech Embedded as E respectively col and E tech It integrates OCR text features (T1), visual features (V), and hierarchical embeddings from knowledge graphs (E). doc E chap ):

[0136]

[0137] E tech =GNN hierarchy (E doc E chap E para E entity ), where SearchImage(c col ImageEncoder is a neural network that converts images into feature vectors (e.g., ResNet for extracting visual features), and GNN is a user search image corresponding to colloquial terms. hierarchy Hierarchical graph neural network, with input knowledge graph multi-layer node embeddings (document layer, chapter layer, paragraph layer, entity layer).

[0138] Next, fault keywords are extracted. First, query parsing is enhanced based on colloquial nodes from a tag-matching database, avoiding the problem of traditional word vectors ignoring domain mapping. Let the word sequence of the user query Q be {w1, w2, ..., w...}. m Candidate fault keywords K are extracted by filtering through a tag comparison library. EditDist is the user's query term w i With spoken name node c col The edit distance, τ is the domain tolerance threshold, for example, setting it to 2 allows "brake" and "brake" to have an edit distance of 1, triggering tag library matching.

[0139] Then, synonyms and / or near-synonyms are expanded. Traditional methods are based only on general word vectors, while this approach uses a bipartite graph structure of a tag lookup library for domain-specific expansion.

[0140] in: These are the technical terms in the tag library that directly map to the colloquial word k (e.g., "accelerator" → "throttle body"). These are related terms obtained through the propagation of "component-fault" relationships in the knowledge graph (e.g., "throttle body" → "throttle position sensor fault").

[0141] Finally, an expanded query request is generated, which means expanding the synonym set S = {s1, s2, ..., s...} n Semantically align with the entity relation layer nodes of the knowledge graph to generate a query vector with structural constraints: Q ext =Transformer kg (S,R rel ), where R rel The relationship types between entities in the knowledge graph serve as structural inputs for the Transform, such as "fault code-solution" and "part-repair procedure".

[0142] After receiving the expanded query request, matching can be performed in the tag comparison library based on fault synonyms and / or fault near-synonyms, that is, matching the expanded fault synonyms s i The optimal professional name is determined by matching it with a tag reference library and using a two-way attention mechanism.

[0143] The attention score is calculated as follows: Where γ is the structural relevance weight, which forces priority matching of professional terms with high structural importance in the knowledge graph, Attn(e col ,e tech ): Two-way attention score, which calculates the matching degree between spoken language embeddings and technical terminology embeddings.

[0144] Then, a search is performed in the knowledge graph based on the major name, i.e., based on the major name. When querying a knowledge graph, both textual data and structural relationships are retrieved simultaneously, thus obtaining document fragment information:

[0145] Text data: via paragraph layer node Epara Extracting contains semantic units

[0146] Structural relationship: via chapter-level node E chap Get Column Layout (R) col ),

[0147] Through document layer node E doc Get cross-page merge relationship (R) page ) TextSearch extracts text data containing technical terms from paragraph-level nodes in the knowledge graph. StructSearch retrieves column layout data from chapter-level and document-level nodes. col ), cross-page merging (R) page Structural relationships such as )

[0148] For example, a user might query: "The accelerator is very slow to respond when driving, and there is no acceleration power."

[0149] AD-RAG analysis: Extracting fault keywords: "slow throttle response", "weak acceleration";

[0150] Matching by tag library: "Slow throttle response" → "Throttle response delay" (direct mapping), "Weak acceleration" → "Insufficient power" (domain expansion);

[0151] Knowledge graph search: Professional names: "Throttle response delay", "Insufficient power";

[0152] Text data: Paragraph layer: "Throttle position sensor voltage detection steps" (including voltage threshold parameters);

[0153] Physical layer: Associated fault code "P0121" and its description "Throttle position sensor signal is unreasonable";

[0154] Structural Relationship: Column Layout: The testing steps and the throttle circuit diagram are located in the same column (R col (Mark the coordinates of the region);

[0155] Page merging: This detection process spans pages 3-8 to 3-10 (R) page Provides continuous page number mapping);

[0156] Output: Structured text: Detection steps arranged in logical order, with automatic merging of content spanning multiple pages;

[0157] Related visual: Schematic diagram of throttle position sensor, with the correspondence between detection points and circuit diagram numbers marked.

[0158] The technical solution provided in this embodiment has the following technical advantages:

[0159] 1. In traditional methods, the tag library is an independent mapping table, which is disconnected from the entity relationships in the knowledge graph (e.g., "accelerator" maps to "throttle body", but cannot be associated with the fault code node of "throttle body" in the knowledge graph). This solution, however, uses a bipartite graph. By directly connecting spoken word nodes with technical terminology nodes in the knowledge graph, a complete link is formed: "spoken description → technical terminology → fault code → repair steps". For example, "brakes becoming soft" → "air ingress into the braking system" → "P0571 fault code" → "brake fluid bleed procedure section". Each link conveys semantics through graph structure relationships, achieving deep coupling between the tag lookup library and the knowledge graph.

[0160] 2. Traditional methods using word vectors trained on general corpora (such as Word2Vec) show a cosine similarity of only 0.65 between "accelerator" and "throttle body". In contrast, this approach ensures accurate matching (similarity 1.0) through direct mapping from a tag library and utilizes knowledge graph structure propagation (GraphProp) to discover deep associations (such as "throttle body" → "throttle position sensor" → "signal anomaly"), expanding the results to include domain-specific terminology chains.

[0161] 3. Traditional searches only return text fragments, ignoring document layout (such as multi-page tables or mixed text and images in columns). This solution, however, uses R during the query process. col and R page Simultaneously, structural relationships are acquired to provide structural semantic support for node embedding in the tag comparison library, ensuring that the "technical term" node carries document layout and hierarchical information. For example, when searching for "brake fluid replacement steps", the cross-page "steps 1-3 (page 20)" and "steps 4-6 (page 21)" are automatically merged, and the corresponding brake system diagram positions in the columns are marked.

[0162] 4. Label Reference Library As a bridge, connecting and Provides domain-specific mapping rules for AD-RAG keyword expansion; Knowledge Graph Hierarchical Embedding (E doc E chap This provides structural semantic support for the node embedding in the tag reference library, ensuring that "technical term" nodes carry document layout and hierarchical information; structural relationships (R) col ,R page This is explicitly invoked during the query phase, so that the search results not only contain text data, but also document structure information that is strongly related to the user's query.

[0163] 5. This application introduces a domain-specific similarity measurement system, such as... It combines textual semantics (DomSim), graph structure association (StrAssoc), and visual feature alignment (VisSim), rather than the traditional single cosine similarity or edit distance, and introduces knowledge graph structure association (StrAssoc) as an index term in the denominator to force priority matching of professional terms that are more closely connected in the graph (e.g., "air infiltration in the braking system" is 3 times more structurally important than "brake fluid problem").

[0164] 6. This application adopts a dynamic knowledge fusion extension mechanism, with bidirectional propagation between the tag library and the knowledge graph (formula SynExt(K)): extended terms are generated through direct mapping (DirectMap) and graph structure propagation (GraphProp), rather than traditional word vector clustering or thesaurus query.

[0165] 7. This application employs structure-aware retrieval output, with structured retrieval functions. It also returns text data and structural relationships (R). col ,R page (This is a list of text elements, not a traditional plain text list.)

[0166] Task classification and reasoning optimization module 4 is used to parse and decompose the extended query request to obtain at least one fault sub-problem sequence, and to perform fault reasoning based on the fault sub-problem sequence and document fragment information through a preset fault reasoning model to obtain fault repair suggestion results.

[0167] In the repair scenario of vehicle engine idling vibration, the system needs to break down the user query "steering wheel vibrates noticeably at idle" into sub-problems such as "ignition system failure", "abnormal fuel injection", and "air intake system leakage". Combining the spark plug replacement steps, fuel injector testing procedures and other document fragments in the repair knowledge graph, the system generates complete repair suggestions through iterative reasoning.

[0168] First, we need to generate a sequence of fault subproblems and define the extended query request Q. ext The corresponding knowledge graph subgraph is Where node N q Includes fault symptoms (e.g., "idling vibration"), potential fault locations (e.g., "spark plugs" and "fuel injectors"), and repair steps (e.g., "checking spark plug clearance"). q This represents the relationship between "phenomenon-fault" and "fault-step". An attention-guided subgraph segmentation algorithm generates a sequence of fault subproblems S = {s1, s2, ..., s...}. m}: Where: R re l represents the relationship type in the knowledge graph (e.g., "cause" or "solution"), W conf The node confidence matrix is ​​determined by both the frequency of repair cases and the structural integrity of document fragments.

[0169] Where CaseFreq(n i The number of times the fault occurred in historical cases is represented by ), StructScore calculates document fragment integrity based on page merging and column layout, GraphSplit represents the attention-guided subgraph segmentation function (domain-specific algorithm, not general graph segmentation), and W... conf (n i Let be the confidence score of the i-th node. This represents the total number of historical repair cases.

[0170] Then, the dependencies between the fault subproblem sequences are modeled, and a Temporally Transformed Graph Convolutional Network (TGCN) is introduced to capture the execution order of the subproblems (e.g., inspecting the circuit before disassembling the parts): Wherein: H t PosEnc(t) represents the hidden state of the t-th subproblem, PosEnc(t) is the positional encoding based on the logical order of maintenance steps (e.g., "fault code reading" must be done before "parts removal"), and GCN is a graph convolutional network layer.

[0171] For example, if a user inputs "the steering wheel vibrates noticeably at idle," the knowledge retrieval enhancement module 3 will parse the query and obtain an extended query request Q. ext ={Idle vibration, steering wheel vibration, malfunction indicator light flashing}, according to Q ext Extracting related subgraphs from knowledge graphs Include:

[0172] Phenomenon node: Idle speed vibration (n sym ), steering wheel vibration (n vib );

[0173] Candidate fault node: Spark plug gap too large (n spk Poor fuel injector atomization (n inj ), intake manifold leakage (n leak );

[0174] Repair procedure step: Check spark plug resistance (n) step1 ), Read fault codes (n step2 ).

[0175] Attention-guided subgraph segmentation algorithm (formula) Combining the frequency of repair cases (e.g., "spark plug failure" accounts for 40% of historical idling vibration cases) and the integrity of the document structure ("spark plug inspection steps" spans pages 3-5, with a structure integrity of 0.92), a sequence of fault sub-problems is generated: S = [s1: read fault code, s2: check spark plug gap, s3: check injector atomization]. This sequence is used to parse and decompose the extended query request to obtain at least one sequence of fault sub-problems.

[0176] Preferably, fault reasoning is performed based on the fault sub-problem sequence and document fragment information using a pre-defined fault reasoning model to obtain fault repair suggestion results, including:

[0177] Based on the sequence of fault sub-problems and document fragment information, fault reasoning is performed using a pre-set fault reasoning model to obtain the first reasoning result;

[0178] When the first reasoning result is incomplete, a supplementary question sequence is obtained based on the first reasoning result;

[0179] Based on the supplementary question sequence and the first reasoning result, fault reasoning is performed through the fault reasoning model to obtain fault repair suggestions.

[0180] First, the completeness of the first inference result needs to be evaluated. The completeness index of the first inference result R1 is defined as the information entropy reduction rate:

[0181] Where InfoGain(n) is the information gain of node n for fault diagnosis, calculated using text data from the maintenance case library and document fragments: (P(n|fault phenomenon) is the conditional probability of the node appearing in the case, TF-IDF measures the importance of the term in the document, R1 is the first inference result, and P(n) represents the prior probability of the node appearing in all cases.

[0182] Then, supplement the reinforcement learning model that generates the problem sequence, that is, when At that time, supplementary questions s are generated using a Deep Q-Network (DQN). sup :s sup =argmax a Q(S t ,a;θ), where state S t Includes the current inference result, the state of the knowledge graph subgraph, and the document structure relationships. Action 'a' is a candidate supplementary question (e.g., "Has the fuel pressure been checked?"). argmax a This indicates selecting the action that maximizes the Q value (supplementary question), Q(S) t,a;θ):Q-function of reinforcement learning, which evaluates the value of state-action pairs, a represents candidate actions (i.e. candidate supplementary questions), θ:Parameters of the Q-network (learned through training).

[0183] The reward function is designed as follows: in, To complement the improved completeness provided by the questions, StructConsistency ensures that the questions are consistent with the document structure, QueryCost represents the cost for maintenance personnel to answer, and γ1, γ2, and γ3 represent the weighting parameters that balance different reward items.

[0184] For example, inputting S along with document fragment information (such as the "Spark Plug Inspection Procedure" section in the Engine Repair Manual, or the injector test data) into the fault reasoning model M reason The model has been trained using historical cases (including the case of "idling vibration → spark plug replacement → troubleshooting") and an updated knowledge graph.

[0185] Input: a sequence of subproblems S, text data T1 of document fragments, and structural relations R;

[0186] Output: First inference result R1 = {Spark plug gap is too large, it is recommended to replace spark plugs}.

[0187] Through the information entropy reduction rate (formula) R1 assessment:

[0188] like (Possibility of missing fuel injectors and intake system), triggering the supplementary issue mechanism.

[0189] Generate supplementary question sequence S based on Deep Q-Network (DQN) sup =[Have you checked the injector atomization status? Is the fault code P0300?] Prioritize issues with high information gain and low repair costs (such as "reading fault codes" without disassembling parts).

[0190] Finally, S sup Combined with R1 input M reason Based on the association edges of "poor injector atomization → idle vibration" in the new knowledge graph (updated through new cases), the final repair suggestion is: R final ={Spark plug gap too large (replace), fuel injector atomization poor (clean), repair steps in sequence: replace spark plug first, then clean fuel injector}.

[0191] Preferably, the fault reasoning model is trained based on vehicle fault query requests and the updated maintenance knowledge graph. The updated maintenance knowledge graph is obtained by updating the maintenance knowledge graph through the knowledge graph and intelligent recommendation module.

[0192] The knowledge graph and intelligent recommendation module is used to obtain repair cases, corresponding fault phenomena, and corresponding solutions, and to add repair cases, fault phenomena, solutions, and corresponding relationships to the repair knowledge graph to update the repair knowledge graph.

[0193] Fault reasoning model M reason The training objective is to minimize the structured loss function, using the repair case-knowledge graph-document structure triple as input.

[0194]

[0195] Wherein: GraphMatchLoss measures the structural similarity between the inference subgraph and the case subgraph (calculated by GNN graph kernel), and StructuralLoss constrains the document structure relationship of the inference results (such as the order of steps across pages) to be consistent with the annotation.

[0196] When new repair cases are added In this case, the knowledge graph is updated and maintained through incremental graph embedding: in:

[0197] ContrastiveUpdate enhances the association between new fault-solution pairs through contrastive learning. part Provide schematic diagrams of the parts involved in the new fault to ensure consistency between the text and images in the updated diagrams.

[0198] The training process of the fault reasoning model first requires the construction of training data, including input data and output data. The input data includes, for example, historical vehicle fault query requests (extended query requests and fault sub-problem sequences after being parsed by the AD-RAG model); the updated maintenance knowledge graph, which includes structured entities (such as "throttle body" and "idle speed motor"), relationships (such as "component-fault phenomenon" and "fault-solution"), attributes (such as fault codes and maintenance step time), and maintenance cases added through the knowledge graph and intelligent recommendation module (fault phenomenon → solution mapping pairs, such as "idle speed shaking + fault code P0507 → cleaning throttle body"). Supplementary question sequences (follow-up questions generated in historical interactions due to incomplete reasoning, such as "has the intake manifold leak been detected?"). The output data includes standardized fault maintenance suggestion results (such as "[1] detect throttle opening signal; [2] clean idle speed motor; [3] reset ECU adaptive data").

[0199] Next, the model architecture needs to be designed, adopting a hybrid architecture of "sequence decomposition - graph structure reasoning - dynamic completion". The first layer is the encoding of fault sub-problem sequences. An improved Transformer model is used to encode the fault sub-problem sequences (such as "Possible causes of idling vibration?" "How to detect the intake system?") to capture semantic dependencies.

[0200] H s =MultiHeadAttention(LayerNorm(Q s W Q LayerNorm(K) s W K LayerNorm(V) s W V ), where Q s ,K s V s W is the query, key, and value matrix of the subproblem sequence. Q W K W V These are trainable parameters.

[0201] The second layer involves embedding knowledge graph relationships. A graph neural network (GNN) is used to embed the structural relationships of the maintenance knowledge graph (such as "throttle body → connection → intake manifold" and "idling vibration → from → intake leakage"), generating a vector representation E of entities and relationships. g ,R g This addresses the issue of traditional methods neglecting the domain-specific semantic gap: in, For a set of relation types, Let u be a neighboring node of relation r. This is the relation and node parameter matrix for the l-th layer.

[0202] The third layer: Dynamic completion reasoning. When the first reasoning result output by the first and second layers is incomplete (e.g., confidence < threshold or missing key steps), reinforcement learning (RL) is used to generate a sequence of supplementary questions, such as: Among them, a t Let r(s,a) be the supplementary question action at step t, r(s,a) be the reward (such as information gain) for choosing question a in state s, γ be the discount factor, and V(s′) be the value function for subsequent states, guiding the model to prioritize questions that can fill knowledge gaps.

[0203] Finally, the model is trained by taking historical fault query requests, knowledge graph entity relationships, and maintenance cases as input, and minimizing the cross-entropy loss between predicted maintenance suggestions and labels: Where N is the number of training samples, K is the number of maintenance step categories, and y i,kFor tags, The model is used to predict probabilities. For the "supplementary question sequence generation" task, the model's follow-up questioning strategy is optimized using a policy gradient algorithm when information is incomplete, improving the completeness of reasoning. When the knowledge graph and intelligent recommendation modules add new repair cases (such as new electric vehicle battery management fault cases), the model is fine-tuned through transfer learning to avoid retraining the entire network and improve efficiency. Where θ represents the parameters of the pre-trained model, and α represents the learning rate. The loss function for the new case.

[0204] The technical solution provided in this implementation has the following technical advantages:

[0205] 1. Traditional formulas (such as TF-IDF and BM25) only process text frequency and cannot capture key information such as "whether the repair steps are complete across pages" and "the probability of the fault point appearing in historical cases." This application introduces repair case frequency (CaseFreq) and document structure score (StructScore) instead of the commonly used term frequency or TF-IDF. It also combines this with cross-page merging specific to vehicle repair (R... page ) and column layout (R col ), quantify the operability of the fault point (e.g., prioritize the "spark plug test" with complete steps in the document rather than the fragmented "intake manifold leak").

[0206] 2. Traditional inference models (such as Naive Bayes and general neural networks) only take text as input, and their loss functions do not involve graph structure or document layout. This leads to problems such as "suggested steps breaking across pages" and "fault points being disconnected from system hierarchy" in the inference results. In this application, the loss function integrates text classification loss, graph structure matching loss, and document structure constraint loss, forcing the inference results to simultaneously satisfy semantic correctness, knowledge graph structure consistency, and document layout integrity. For example, the suggestion of "replacing spark plugs" needs to be associated with the hierarchical relationship of "spark plug → ignition system" in the knowledge graph, and at the same time correspond to the structural relationship of "cross-page steps in Chapter 3, Section 2" in the document.

[0207] 3. Traditional methods generate supplementary questions through rule engines or fixed templates, which cannot dynamically balance information value and operational costs, and are not aware of document structure (e.g., they cannot associate the specific location of "the detection steps are on page 5, column B of the manual"). This application, however, uses reinforcement learning (DQN) to generate supplementary questions, and the reward function integrates information gain. Structural consistency and query cost ensure that supplementary issues are both efficient and logically sound (e.g., prioritizing "reading fault codes" over "disassembling the engine"). And state S tIt includes knowledge graph subgraph states and document structure relationships, making the reasoning process explicitly dependent on the hierarchical structure of vehicle maintenance (e.g., "engine system → ignition subsystem → spark plug components").

[0208] Accordingly, this application also provides a method for parsing vehicle after-sales maintenance documents and acquiring maintenance knowledge based on AD-RAG.

[0209] Figure 2 This is a flowchart illustrating a method for parsing vehicle after-sales maintenance documents and acquiring maintenance knowledge based on AD-RAG, provided as an embodiment of this application. Figure 2 As shown, the AD-RAG-based method for parsing vehicle after-sales maintenance documents and acquiring maintenance knowledge includes:

[0210] 101. Obtain vehicle after-sales maintenance documentation information, and perform layout analysis and document recognition on the after-sales maintenance documentation information to obtain structured data, which includes the text data and structural relationships of the after-sales maintenance documentation information.

[0211] There are various ways to obtain vehicle after-sales maintenance documentation. This can include scanning, obtaining photos, direct electronic document transmission, or directly accessing data sources, such as retrieving maintenance documents from the automaker via a network connection. After-sales maintenance documentation includes text, images, circuit diagrams, tables, etc., and may include vehicle and component maintenance manuals, technical service bulletins, fault diagnosis flowcharts, fault records, and historical repair records.

[0212] Layout analysis and document recognition of after-sales maintenance documents involve analyzing the layout and recognizing the original documents to extract the document's chapter structure, paragraph content, charts, and other information, thereby forming structured data. In this process, relevant document content can be obtained while preserving the hierarchical relationships within the document, thus acquiring structured data.

[0213] 102. Establish associations between text data and structural relationships in structured data to obtain a maintenance knowledge graph. The maintenance knowledge graph includes nodes that reflect different granularities of information in the text data and edges that connect the nodes. The edges of the nodes are used to reflect the structural relationships between the nodes.

[0214] Nodes with different granularities of information refer to nodes in text data that represent information at different granularities, such as the entire document, chapters, and paragraphs. These nodes are connected by edges, thus revealing the structural relationships between them. By establishing associations between text data and structural relationships, a vast network of relationships between nodes can be obtained, which is essentially a knowledge graph. For example, chapters on the same topic can be connected to different sections or paragraphs because they relate to the same topic. Of course, this topic could also be keywords, etc., and the specific settings can be configured according to the actual situation.

[0215] 103. Receive the user's vehicle fault query request, and parse and expand the vehicle fault query request based on the AD-RAG model to obtain an expanded query request. Then, perform a query on the maintenance knowledge graph based on the expanded query request to obtain document fragment information, which includes text data and structural relationships corresponding to the expanded query request.

[0216] When a user's vehicle malfunctions and requires a query, the system can accept vehicle malfunction query requests from the user, including the specific malfunction. The system then uses the AD-RAG model to parse and expand these requests. For example, if the received malfunction is "battery dead," it can be parsed into technical terms like "low battery power" and further expanded to include "low battery voltage" or "low 12V battery voltage," resulting in an expanded query request. Based on this expanded query request, the system searches the maintenance knowledge graph for matching document data, obtaining the corresponding document fragment information. This document fragment information includes text data and its structural relationships. The knowledge graph contains vehicle-related technical terminology tags; these tags allow users to find the document content corresponding to the nodes, i.e., the document fragment data.

[0217] 104. Parse and decompose the extended query request to obtain at least one sequence of fault sub-problems. Based on the sequence of fault sub-problems and document fragment information, perform fault reasoning through a preset fault reasoning model to obtain fault repair suggestions.

[0218] When the extended query request is a simple request, such as simply querying a parameter value or a single query step, the required answer can be obtained directly through retrieval. However, if the request is more complex (such as comprehensive fault diagnosis involving multiple symptoms or information summarization across multiple documents), then the request needs to be parsed and decomposed.

[0219] For example, if the request is "the steering wheel vibrates noticeably at idle", parsing and decomposing it can identify sub-problems such as "ignition system failure", "abnormal fuel injection", and "air intake system leakage", thus obtaining at least one sequence of fault sub-problems.

[0220] Since the document fragment information has already been obtained, and this document fragment information includes the relevant information required for the fault sub-problem sequence, the fault sub-problem sequence and document fragment information are then fed into the preset fault reasoning model for fault reasoning. This will yield the corresponding reasoning results, which are the fault repair suggestions.

[0221] Through the above methods, the vehicle after-sales maintenance document parsing and maintenance knowledge acquisition method based on AD-RAG provided in this application embodiment can convert vehicle after-sales documents into digital data. During the document retrieval process, relevant personnel do not need to consult paper documents, greatly improving the speed of document retrieval. Simultaneously, it can analyze and identify the layout of structures and circuit diagrams in maintenance documents, enabling relevant personnel to conduct more comprehensive document searches by combining relevant diagrams and tables. Furthermore, it associates textual and structural data in after-sales document information. When a fault query request is received, relevant documents can be quickly found in the maintenance knowledge graph, thereby improving the efficiency of document retrieval. At the same time, through question parsing and expansion, it can better meet the retrieval needs of the maintenance field. Finally, by parsing and decomposing the query request and using a fault reasoning model, more accurate fault repair suggestions can be obtained, improving the efficiency of maintenance knowledge acquisition.

[0222] It should be understood that the present invention is not limited to the processes and structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is limited only by the appended claims.

Claims

1. An AD-RAG based vehicle after-sales maintenance document analysis and maintenance knowledge acquisition system, characterized in that, include: The document parsing module is used to obtain after-sales maintenance document information of the vehicle, and to perform layout analysis and document recognition on the after-sales maintenance document information to obtain structured data. The structured data includes the text data and structural relationships of the after-sales maintenance document information. The knowledge organization module is used to establish associations between the text data and structural relationships in the structured data to obtain a maintenance knowledge graph. The maintenance knowledge graph includes nodes that reflect different granularities of information of the text data and edges that connect the nodes. The edges of the nodes are used to reflect the structural relationships between the nodes. The knowledge retrieval enhancement module is used to receive a user's vehicle fault query request, parse and expand the vehicle fault query request based on the AD-RAG model to obtain an expanded query request, and perform a query in the maintenance knowledge graph based on the expanded query request to obtain document fragment information, the document fragment information including text data and structural relationships corresponding to the expanded query request; The task decomposition and reasoning optimization module is used to parse and decompose the extended query request to obtain at least one sequence of fault sub-problems, and to perform fault reasoning based on the sequence of fault sub-problems and the document fragment information through a preset fault reasoning model to obtain fault repair suggestion results. The document parsing module includes: An optical character recognition unit is used to recognize the after-sales maintenance document information and convert the recognized after-sales maintenance document information into data to obtain first text data. The layout analysis unit is used to perform layout analysis on the after-sales maintenance document information to obtain the structural relationship of the document content of the after-sales maintenance document information; A visual language generation unit is used to extract the display image of the after-sales maintenance document information and annotate the display image based on a pre-trained visual language model to obtain second text data. Based on the optical character recognition unit, the layout analysis unit, and the visual language generation unit, a deep coupling of recognition, structure, and semantics is formed. The optical character recognition unit, on the basis of traditional text recognition, injects layout structure and visual semantics to perform contextual error correction and generates first text data with position information. The layout analysis unit constructs a cross-modal graph structure, integrates the text features, visual semantic image features, and position of the first text data, and outputs a structural relationship that includes columns, sequence, and page cross-page relationships; The visual language unit utilizes a knowledge base and structural relationships to perform domain enhancement and cross-validation on the image-annotated second text data, making the first and second text data consistent in semantics and structure. Through a fusion model, the first and second text data are integrated under the constraints of structural relationships to generate text data that conforms to human reading logic, is structurally complete, and semantically accurate. The text data includes the first text data and the second text data; The after-sales service documentation information is subjected to document recognition, and the document content of the recognized after-sales service documentation information is digitized to obtain the first text data, including: The after-sales maintenance document information is identified, and the identified after-sales maintenance document information is fed into a pre-trained large language segmentation model for document segmentation to obtain at least one document block. Each document block includes a complete semantic unit and a document topic corresponding to the semantic unit. The document content of the document block is digitally converted to obtain the first text data; The input to the large language chunking model includes the initial text generated by OCR, the position coordinate matrix of the optical character recognition unit, and the structural relationship of the layout analysis unit. The position coordinate matrix of the optical character recognition unit is used to record the spatial position information of each character / text block in the document to characterize the physical layout features of the document. The structural relationship of the layout analysis unit extracts the semantic structure features of the document through the column adaptive algorithm and the logical reading order model. The physical layout features include at least the two-dimensional coordinates of the character / text block, column affiliation, and page cross-page representation. The semantic structure features include at least the hierarchical structure of the document, column association, and page cross-page continuity. The large language chunking model calculates the chunking boundaries for document chunking in the initial text based on a context-aware chunking weight function, including: extracting the context embedding vector of each position in the initial text based on deep semantic encoding of the large language chunking model, and determining whether the current position is at the semantic unit boundary based on topic modeling; calculating the chunking weight function based on the context embedding vector, preset spatial position weights and logical structure weights, and calculating the chunking boundaries of the document chunks based on the chunking weight function; The spatial location weight is used to reflect the physical location of the text block in the layout, and the logical structure weight is used to calculate the matching degree between the text and the structure through the structural relationship obtained by layout analysis.

2. The vehicle after-sales maintenance document parsing and maintenance knowledge acquisition system based on AD-RAG according to claim 1, characterized in that, The fault reasoning model is trained based on vehicle fault query requests and an updated maintenance knowledge graph. The updated maintenance knowledge graph is obtained by updating the maintenance knowledge graph through a knowledge graph and intelligent recommendation module. The knowledge graph and intelligent recommendation module is used to acquire repair cases, corresponding fault phenomena, and corresponding solutions, and add the repair cases, fault phenomena, solutions, and corresponding relationships to the repair knowledge graph to update the repair knowledge graph.

3. The vehicle after-sales maintenance document parsing and maintenance knowledge acquisition system based on AD-RAG according to claim 1, characterized in that, The maintenance knowledge graph includes: Document layer nodes include complete data of the after-sales maintenance document information; Chapter-level nodes include the chapter structure data of the after-sales maintenance document information; Paragraph layer nodes include paragraph data of the after-sales maintenance document information; Entity relationship layer nodes include vehicle part entity data; The document layer nodes, chapter layer nodes, paragraph layer nodes, and entity relationship layer nodes are connected by edges to establish structural relationships between corresponding nodes in the maintenance knowledge graph.

4. The vehicle after-sales maintenance document parsing and maintenance knowledge acquisition system based on AD-RAG according to claim 1, characterized in that, The knowledge retrieval enhancement module is also used to obtain vehicle-related professional names and corresponding colloquial names, and to establish a tag comparison library based on the professional names and colloquial names, and to establish an association between the tag comparison library and the maintenance knowledge graph. The AD-RAG model is used to parse and expand the vehicle fault query request to obtain an expanded query request, including: The method describes parsing the vehicle fault query request based on the AD-RAG model to obtain fault keywords; Expand the fault keywords to obtain fault synonyms and / or fault near-synonyms; Based on the fault synonyms and / or fault near-synonyms, an extended query request is obtained; The process of querying the maintenance knowledge graph based on the extended query request yields document fragment information, including: The fault synonyms and / or fault near-synonyms are matched with the tag comparison library to obtain the corresponding professional names, which are used as the query names; Based on the query name, a search is performed in the maintenance knowledge graph to obtain document fragment information.

5. The vehicle after-sales maintenance document parsing and maintenance knowledge acquisition system based on AD-RAG according to claim 1, characterized in that, The process of performing fault reasoning based on the fault sub-problem sequence and the document fragment information using a preset fault reasoning model to obtain fault repair suggestion results includes: Based on the sequence of fault sub-problems and the document fragment information, fault reasoning is performed using a preset fault reasoning model to obtain a first reasoning result; When the first reasoning result is incomplete, a supplementary question sequence is obtained based on the first reasoning result; Based on the supplementary question sequence and the first reasoning result, fault reasoning is performed through the fault reasoning model to obtain fault repair suggestion results.

6. A method for parsing vehicle after-sales maintenance documents and acquiring maintenance knowledge based on AD-RAG, characterized in that, include: The system acquires vehicle after-sales maintenance documentation information, performs layout analysis and document recognition on the after-sales maintenance documentation information to obtain structured data, which includes the text data and structural relationships of the after-sales maintenance documentation information. A relationship is established between the text data and structural relationships in the structured data to obtain a maintenance knowledge graph. The maintenance knowledge graph includes nodes that reflect different granularities of information of the text data and edges that connect the nodes. The edges of the nodes are used to reflect the structural relationships between the nodes. The system receives a user's vehicle fault query request, parses and expands the vehicle fault query request based on the AD-RAG model to obtain an expanded query request, and performs a query on the maintenance knowledge graph based on the expanded query request to obtain document fragment information, which includes text data and structural relationships corresponding to the expanded query request. The extended query request is parsed and decomposed to obtain at least one sequence of fault sub-problems. Based on the sequence of fault sub-problems and the document fragment information, fault reasoning is performed through a preset fault reasoning model to obtain fault repair suggestion results. The structured data is obtained based on a document parsing module, which includes: An optical character recognition unit is used to recognize the after-sales maintenance document information and convert the recognized after-sales maintenance document information into data to obtain first text data. The layout analysis unit is used to perform layout analysis on the after-sales maintenance document information to obtain the structural relationship of the document content of the after-sales maintenance document information; A visual language generation unit is used to extract the display image of the after-sales maintenance document information and annotate the display image based on a pre-trained visual language model to obtain second text data. Based on the optical character recognition unit, the layout analysis unit, and the visual language generation unit, a deep coupling of recognition, structure, and semantics is formed. The optical character recognition unit, on the basis of traditional text recognition, injects layout structure and visual semantics to perform contextual error correction and generates first text data with position information. The layout analysis unit constructs a cross-modal graph structure, integrates the text features, visual semantic image features, and position of the first text data, and outputs a structural relationship that includes columns, sequence, and page cross-page relationships; The visual language unit utilizes a knowledge base and structural relationships to perform domain enhancement and cross-validation on the image-annotated second text data, making the first and second text data consistent in semantics and structure. Through a fusion model, the first and second text data are integrated under the constraints of structural relationships to generate text data that conforms to human reading logic, is structurally complete, and semantically accurate. The text data includes the first text data and the second text data; The after-sales service documentation information is subjected to document recognition, and the document content of the recognized after-sales service documentation information is digitized to obtain the first text data, including: The after-sales maintenance document information is identified, and the identified after-sales maintenance document information is fed into a pre-trained large language segmentation model for document segmentation to obtain at least one document block. Each document block includes a complete semantic unit and a document topic corresponding to the semantic unit. The document content of the document block is digitally converted to obtain the first text data; The input to the large language chunking model includes the initial text generated by OCR, the position coordinate matrix of the optical character recognition unit, and the structural relationship of the layout analysis unit. The position coordinate matrix of the optical character recognition unit is used to record the spatial position information of each character / text block in the document to characterize the physical layout features of the document. The structural relationship of the layout analysis unit extracts the semantic structure features of the document through the column adaptive algorithm and the logical reading order model. The physical layout features include at least the two-dimensional coordinates of the character / text block, column affiliation, and page cross-page representation. The semantic structure features include at least the hierarchical structure of the document, column association, and page cross-page continuity. The large language chunking model calculates the chunking boundaries for document chunking in the initial text based on a context-aware chunking weight function, including: extracting the context embedding vector of each position in the initial text based on deep semantic encoding of the large language chunking model, and determining whether the current position is at the semantic unit boundary based on topic modeling; calculating the chunking weight function based on the context embedding vector, preset spatial position weights and logical structure weights, and calculating the chunking boundaries of the document chunks based on the chunking weight function; The spatial location weight is used to reflect the physical location of the text block in the layout, and the logical structure weight is used to calculate the matching degree between the text and the structure through the structural relationship obtained by layout analysis.

Images