Vehicle damage information processing method and device, medium and equipment

Abstract

The application discloses a vehicle damage information processing method, device, medium and equipment, relates to the technical field of artificial intelligence, and is applied to the medical insurance field. The method comprises the following steps: performing initial identification on vehicle damage related files provided by a repair factory based on a multi-modal analysis model to obtain initial identification information; matching the initial identification information with a preset knowledge base; generating a constraint prompt file according to a matching result and a preset constraint module; performing vehicle damage condition identification based on the constraint prompt file and the vehicle damage related files by a large language model to obtain effective damage information; obtaining maintenance information of effective accessories in the effective damage information from a preset maintenance knowledge base; determining an effective maintenance scheme based on the maintenance information of the effective accessories; generating an initial processing file; performing verification on the initial processing file; and taking the initial processing file that passes the verification as a final processing file. The application improves the accuracy and compliance of vehicle damage related file processing in a repair factory.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, medium and device for processing vehicle damage information. Background Technology

[0002] With the deepening development of insurance technology, utilizing artificial intelligence to auto insurance claims automation and intelligence has become a core direction for cost reduction and efficiency improvement in the industry. Documents provided by repair shops recording vehicle damage information (such as damage assessment reports) serve as key evidence for insurance companies to price accident vehicle claims. Insurance companies need to verify and process the vehicle damage information provided by repair shops.

[0003] The existing method for confirming and processing vehicle damage information is to use a large model to parse the vehicle damage information. The large model first recognizes the text in the vehicle damage-related documents, and then performs semantic parsing on the recognized text to obtain key information such as the name, quantity, labor hours, and unit price of the damaged parts.

[0004] However, in order to make the context logical, the large model will automatically deduce and generate a set of seemingly self-consistent but completely unrealistic accessory and price data based on the incorrect text recognition results, resulting in low accuracy and compliance of the vehicle damage information processing results. Summary of the Invention

[0005] In view of this, this application provides a method, apparatus, medium and equipment for processing vehicle damage information, which solves the problems of low accuracy and low compliance of existing methods for processing vehicle damage information.

[0006] According to one aspect of this application, a method for processing vehicle damage information is provided, the method comprising: Obtain vehicle damage-related documents provided by the repair shop, input the vehicle damage-related documents into the trained multimodal analytical model to obtain initial identification information, wherein the initial identification information includes the vehicle model, the name of at least one initially identified part and its corresponding work time keywords; The vehicle model and the name of each initially identified part, along with their corresponding working hour keywords, are matched with a preset knowledge base. A constraint prompt file is generated based on the name of the re-identified part, the reference standard working hours, and the reference price range. The vehicle damage-related files and the constraint prompt file are input into the trained large language model to obtain valid damage information, which includes valid parts. Based on the valid damage information, repair information for valid parts is obtained from a preset repair knowledge base. Based on the vehicle model and the repair information for the valid parts, a valid repair plan is determined. Based on the valid damage information and the determined valid repair plan, an initial processing file is generated. The initial processing file is verified based on preset rules, and the initial processing file that passes the verification is used as the final processing file.

[0007] According to another aspect of this application, a vehicle damage information processing apparatus is provided, the apparatus comprising: The initial identification module is used to obtain vehicle damage-related documents provided by the repair shop, input the vehicle damage-related documents into the trained multimodal parsing model, and obtain initial identification information, wherein the initial identification information includes the vehicle model, the name of at least one initially identified part and its corresponding work time keywords; The re-identification module is used to match the vehicle model and the name of each initially identified part and its corresponding working time keywords with a preset knowledge base. Based on the name of the matched re-identified part, the reference standard working time and the reference price range, a constraint prompt file is generated. The vehicle damage-related files and the constraint prompt file are input into the trained large language model to obtain effective damage information, which includes effective parts. The solution determination module is used to obtain repair information of valid parts from a preset repair knowledge base based on the valid damage information, determine a valid repair solution based on the vehicle model and the repair information of the valid parts, and generate an initial processing file based on the valid damage information and the valid repair solution. The verification module is used to verify the initial processing file based on preset rules, and the initial processing file that passes the verification is used as the final processing file.

[0008] According to another aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described method for processing vehicle damage information.

[0009] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described method for processing vehicle damage information.

[0010] By employing the above technical solution, this application provides a method, apparatus, medium, and equipment for processing vehicle damage information. The method involves inputting vehicle damage-related documents provided by a repair shop into a trained multimodal analytical model to obtain initial identification information. The vehicle model, the name of at least one initially identified part, and its corresponding labor time keywords from the initial identification information are matched with a preset knowledge base. A constraint prompt file is generated based on the name of the matched re-identified part, reference standard labor time, and reference price range. The vehicle damage-related documents and the constraint prompt file are then input into a trained large language model to obtain valid damage information. Based on the valid damage information, repair information for valid parts is obtained from a preset repair knowledge base. An effective repair plan is determined based on the vehicle model and the repair information for valid parts, generating an initial processing file. The initial processing file is validated based on preset rules, and the validated initial processing file becomes the final processing file. A constraint prompt file is generated by matching the initially identified parts with the knowledge base. The large language model performs vehicle damage identification under the constraints of the constraint prompt file, without fabricating part names and prices. This improves the accuracy and compliance of vehicle damage information processing, increases the efficiency of insurance company claims, and significantly improves user trust and service satisfaction.

[0011] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A flowchart illustrating a method for processing vehicle damage information according to an embodiment of this application is shown. Figure 2 This paper illustrates another flowchart of a method for processing vehicle damage information provided in an embodiment of this application. Figure 3 This illustration shows another flowchart of a method for processing vehicle damage information provided in an embodiment of this application; Figure 4 This illustration shows a structural schematic diagram of a vehicle damage information processing device provided in an embodiment of this application; Figure 5 A schematic diagram of the device structure of a computer device provided in an embodiment of this application is shown.

[0013] in, Figure 4In the Chinese section: 402 - Initial Identification Module; 404 - Re-Identification Module; 406 - Scheme Determination Module; 408 - Verification Module. Detailed Implementation

[0014] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0015] The vehicle damage information processing method provided in this invention can be applied in the application environment of an insurance company's server with vehicle damage information processing functions for repair shops.

[0016] After a vehicle accident, the repair shop staff generates vehicle damage-related documents recording the damage (including damaged parts, corresponding labor hours, repair prices, etc.). Because the service platforms used by the repair shop and the insurance company are not connected, the repair shop's vehicle damage-related documents contain a significant amount of paper material. The insurance company's service platform needs to verify the pricing of these paper materials. Furthermore, the part names in the repair shop's service system may differ from those in the insurance company's system; for example, the repair shop might add the vehicle's brand or model name before the part name. The repair shop's repair quote calculation method also differs from the insurance company's. Therefore, the insurance company's staff needs to verify and confirm the vehicle damage-related documents provided by the repair shop according to the insurance company's pricing method and part names.

[0017] Existing methods for verifying and processing vehicle damage-related documents in repair shops employ large-scale models to identify these documents. However, these general models often misidentify textual content, leading to the fabrication of false parts information or price data, severely reducing the accuracy and compliance of vehicle damage information processing and verification. This application generates a constraint prompt document. Under the constraints of this document, the large-scale model will not fabricate false parts information or price data, ensuring the identification of genuine, valid parts. It retrieves repair information for valid parts from a pre-defined repair knowledge base, determines an effective repair plan based on this information, generates an initial processing document, and validates the initial processing document according to pre-defined rules. The validated initial processing document becomes the final processing document, significantly improving the accuracy of vehicle damage-related document verification and processing in repair shops. The server-side can be implemented using a dedicated server or a server cluster consisting of multiple servers. The following detailed description of specific embodiments further illustrates this invention.

[0018] This embodiment provides a method for processing vehicle damage information, such as... Figure 1 As shown, the method includes: 102: Obtain vehicle damage-related documents provided by the repair shop, input the vehicle damage-related documents into the trained multimodal analytical model to obtain initial identification information, which includes the vehicle model, the name of at least one initially identified part and its corresponding labor time keywords; 104: Match the vehicle model and the name of each initially identified part, along with their corresponding working hour keywords, with the preset knowledge base. Generate a constraint prompt file based on the name of the re-identified part, the reference standard working hour, and the reference price range. Input the vehicle damage-related files and the constraint prompt file into the trained large language model to obtain valid damage information, which includes valid parts. 106: Based on valid damage information, retrieve repair information for valid parts from a pre-set repair knowledge base, determine a valid repair plan based on the vehicle model and the repair information for valid parts, and generate an initial processing file based on the valid damage information and the valid repair plan; 108: The initial processing file is validated based on preset rules, and the initial processing file that passes the validation is used as the final processing file.

[0019] Specifically, because the service systems used by repair shops and insurance companies are different, vehicle damage-related documents from repair shops cannot be directly transmitted to the service systems used by insurance companies. Vehicle damage-related documents from repair shops include both text and images. Images include images of the damaged vehicle at the scene, paper materials, etc. First, a multimodal analysis model is used to perform initial analysis and recognition of the text and images in the vehicle damage-related documents to determine the vehicle model of the damaged vehicle, at least one initially identified part, and the labor time keywords corresponding to the repair of the initially identified part.

[0020] Because the model may fabricate information when it makes a mistake in text recognition, constraint prompts are added when performing secondary recognition on vehicle damage-related documents.

[0021] Based on the initially identified vehicle model and parts, along with their corresponding labor hour keywords, a matching process is performed in a pre-defined knowledge base to obtain the name, reference standard labor hours, and reference price range of the re-identified parts that match the initially identified parts. A constraint prompt file is then generated based on the name, reference standard labor hours, and reference price range of the re-identified parts. This constraint prompt file constrains the large language model to identify vehicle damage-related documents, ensuring there is no issue of fabrication, and thus obtaining valid damage information. The valid damage information includes the vehicle model, the name, labor hours, and price of the re-identified valid parts.

[0022] Based on the name of the valid parts, repair information for those parts is retrieved from a pre-defined repair knowledge base. This information includes repair items, standard labor hours, and repair shop details. Based on this information, a valid repair plan is determined, generating an initial processing document. To avoid anomalies or inconsistencies in the initial processing document, it is validated. Only the validated initial processing document is used as the final approved document. If the initial processing document fails validation, the vehicle damage-related documents are re-validated.

[0023] The vehicle damage information processing method of this application can be executed by a processor or a server. For the insurance company's service platform, which is set up on the server, when the user receives the vehicle damage-related documents from the repair shop's staff, the user inputs the vehicle damage-related documents into the server. The server executes the above-mentioned vehicle damage information processing method to process the vehicle damage-related documents from the repair shop and generate the final processed document.

[0024] This application provides a payment receipt processing method. Compared with existing technologies, it inputs vehicle damage-related documents provided by the repair shop into a trained multimodal parsing model to obtain initial identification information. The vehicle model, the name of at least one initially identified part, and its corresponding labor time keywords from the initial identification information are matched with a preset knowledge base. A constraint prompt file is generated based on the name of the matched re-identified part, reference standard labor time, and reference price range. The vehicle damage-related documents and the constraint prompt file are then input into a trained large language model to obtain valid damage information. Based on the valid damage information, repair information for valid parts is obtained from a preset repair knowledge base. An effective repair plan is determined based on the vehicle model and the repair information for valid parts, generating an initial processing file. The initial processing file is validated based on preset rules, and the validated initial processing file becomes the final processing file. A constraint prompt file is generated by matching the initially identified parts with the knowledge base. The large language model performs vehicle damage identification under the constraints of the constraint prompt file, without fabricating part names. This improves the accuracy and compliance of vehicle damage information processing, increases the efficiency of insurance company claims processing, and significantly improves user trust and service satisfaction.

[0025] The insurance company's server terminal uses the vehicle damage information processing method of this application to input the vehicle damage-related documents provided by the repair shop into a trained multimodal parsing model to obtain initial identification information. The vehicle model, the name of at least one initially identified part, and its corresponding labor time keywords from the initial identification information are matched with a preset knowledge base. A constraint prompt file is generated based on the name of the matched re-identified part, reference standard labor time, and reference price range. The vehicle damage-related documents and constraint prompt file are then input into a trained large language model to obtain valid damage information. Based on the valid damage information, repair information for valid parts is obtained from a preset repair knowledge base. Based on the vehicle model and the repair information for valid parts, a valid repair plan is generated, and an initial processing file is generated. The initial processing file is verified based on preset rules, and the verified initial processing file is used as the final processing file, thus improving the accuracy and compliance of the repair shop's vehicle damage-related document processing.

[0026] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, another method for processing vehicle damage information is provided, such as... Figure 2 As shown, the preset knowledge base includes a parts knowledge base and a standard labor hour base. The vehicle model and the name of each initially identified part, along with its corresponding labor hour keywords, are matched against the preset knowledge base. Based on the matched actual part name, reference standard labor hours, and reference price range, a constraint prompt file is generated, including: 202: Obtain the names of all candidate parts that match the vehicle model from the parts knowledge base. For each initially identified part, calculate the similarity between the name of the initially identified part and the name of each candidate part. When the maximum similarity value is greater than the preset similarity threshold, use the name of the candidate part corresponding to the maximum similarity value as the name of the re-identified part. 204: Based on the name of each re-identified part and its corresponding time keywords, the time is matched in the standard time database to obtain the reference standard time and reference price range of the re-identified part; 206: Obtain the preset constraint template, which includes knowledge constraints, task constraints, and rule constraints; 208: Fill in the vehicle model, the name of each re-identification part, the reference standard working hours and reference price range of each re-identification part into the knowledge constraints, and generate a constraint prompt file.

[0027] Specifically, a pre-defined parts knowledge base is connected to the parts supplier system. This knowledge base stores parts information for all vehicle models, containing millions of parts. It periodically pulls the latest parts data from the parts suppliers' ERP systems and monitors the message queue in real time. When a new or modified part is added, it processes the request based on the monitored messages. By injecting a structured knowledge base (including millions of parts codes, labor hour standards, repair shop policies, etc.), the model acquires real-world prior knowledge.

[0028] Based on the accessory knowledge base, semantic similarity retrieval is performed on the name of each initially identified accessory, and the similarity is calculated. When the maximum similarity value is greater than the preset similarity threshold, the name of the candidate accessory corresponding to the maximum similarity value is used as the name of the re-identified accessory.

[0029] The standard labor hour database stores the labor hour information used for each part in the parts knowledge base using different repair solutions. The standard labor hour database stores the part name, repair item (e.g., replacing the front bumper), standard labor hours (hours), difficulty coefficient [e.g., 0.8, 1.5], applicable vehicle model, regional adjustment {e.g., first-tier cities: 1.2, second-tier cities: 1.0, ...}, skill level (beginner, intermediate, advanced), and the repair price corresponding to the labor hours.

[0030] Matching labor time standards based on part name, repair item, vehicle model, and region, using a three-level matching logic: First, perform precise matching: query the standard labor hour database to find labor hour standards that are completely consistent with the part name and repair items and are suitable for the current vehicle model. If the match is successful, calculate the reference standard labor hour by adding the regional adjustment coefficient to the labor hour standard. Then, add the regional price adjustment coefficient to the repair price corresponding to the labor hour and return the reference standard labor hour and reference price range. Next, fuzzy matching is performed: when there are no precise results, all working hour standards are read, and the matching degree is calculated through text similarity. If the similarity is higher than the preset threshold, the optimal matching item is selected and the regional coefficient is added to return the reference standard working hours and reference price range. Finally, there is rule-based interpolation inference: when fuzzy matching also fails, the appropriate working hour standard is automatically calculated based on the rules of similar repair projects, and the reference standard working hours and reference price range are returned.

[0031] To address the "illusion" problem (i.e. fabricating non-existent parts or prices) that can easily occur when large models process vehicle damage-related files, a constraint template was set up, which includes knowledge constraints, task constraints, and rule constraints.

[0032] Task constraints are used to constrain the large language model to identify vehicle damage information from vehicle damage-related documents in the role of a car insurance claims expert, and to output the identified vehicle damage information in a preset format.

[0033] Knowledge constraints are used to define the name of the re-identified part, the reference standard working hours for each re-identified part, and the reference price range as a knowledge pool. This constrains the large language model to identify vehicle damage based on the knowledge pool and related documents. The knowledge constraints explicitly tell the model "please only use the information in the knowledge pool to make judgments", forcing the model to find the answer within the given range, rather than generating it out of thin air.

[0034] Rule constraints are used to guide the large language model in identifying vehicle damage based on knowledge constraints from relevant documents. When the knowledge constraints do not support the identification of any part for damage assessment, it is marked as undetermined. The rule constraints include hard rules to prevent the model from guessing when faced with ambiguous or unrecognizable content, thus ensuring the reliability of the damage assessment results. Examples include: "Fabricating any part names, prices, or labor hour information is strictly prohibited," and "If a field cannot be determined, enter 'Manual review required'."

[0035] In summary, the constraint template strictly prohibits the model from fabricating any part names, prices, or labor time information. If a field cannot be determined, fill in "requires manual review".

[0036] Existing models risk generating fictitious content when processing unstructured documents, such as incorrectly fabricating parts names or pricing information, severely impacting the reliability of damage assessment. This application introduces constraint prompts to guide the large model to focus on field extraction tasks rather than arbitrary generation. This "knowledge constraint + task guidance" approach significantly reduces the rate of misinterpretations, lowering the error information generation rate from the industry average of 6.2% to below 0.8% in testing. The accuracy of key field extraction from vehicle damage-related documents is improved from the industry average of 82% to 92.7%, and the large model's misinterpretation rate is reduced from 6.2% to 0.8%, significantly enhancing data reliability.

[0037] In one embodiment, such as Figure 3 As shown, based on the vehicle model and available parts repair information, an effective repair plan is determined, including: 302: Based on the vehicle model, search for the global map corresponding to the vehicle model in the preset vehicle map. The global map includes all component nodes and their associated work hour nodes. 304: Based on the names of valid parts in the valid damage information, filter the parts nodes and associated work hour nodes related to the valid parts in the global graph, add a repair shop node to the filtered graph, and associate the repair shop node with the parts node as a sub-graph; 306: Input the sub-map into the preset prediction model to obtain the effective repair plan for each effective part. The effective repair plan includes the best repair method and its corresponding labor hours and benchmark price. 308: Obtain the regional price adjustment coefficient corresponding to the region where the repair shop is located, and use the product of the regional price adjustment coefficient and the benchmark price as the repair price of the effective parts.

[0038] Specifically, the preset vehicle map includes a global map corresponding to all vehicle models. Each vehicle model's global map includes parts nodes and work hour nodes. These nodes (parts, work hours, and vehicles) are all assigned unique identifiers to ensure that parts IDs, work hour IDs, and vehicle VIN codes are not duplicated. Some global maps also include repair shop nodes, and correspondingly, repair shop IDs are also unique.

[0039] The parts node stores the parts number, name, category, base price, compatible vehicle models, supplier, and creation time; the work hour node stores the work hour number, repair item, repair type, standard work hours, difficulty level, and required repair skills; the vehicle node stores the vehicle VIN code, vehicle model, brand, year of manufacture, and region; and the repair shop node stores the shop number, shop name, shop type, region, discount level, and service rating.

[0040] Based on the part nodes and associated work hour nodes related to the valid parts, a local subgraph is extracted from the global graph corresponding to the vehicle model. If there is no repair shop node in the global graph, a repair shop node is added to the local subgraph, and the repair shop node is associated with the part node to obtain the subgraph. If there is a repair shop node in the global graph, there is no need to add a repair shop node to the local subgraph, and the local subgraph is directly used as the subgraph.

[0041] The subgraph is input into a preset prediction model. The prediction model traverses all associated edges in the subgraph, constructs an undirected edge index bidirectionally, and reads the confidence level of each relationship as the edge weight. Based on node features, edge index, and edge weight, pricing inference is performed to obtain the best repair plan, corresponding labor hours, and base price for each valid part. The base price includes the repair shop's discount. The final repair price is calculated based on the corresponding regional price adjustment coefficient matched according to the repair shop's location.

[0042] The final repair unit price = base price × regional price adjustment coefficient.

[0043] The fairness score for this pricing is calculated based on fairness characteristics, vehicle brand, and regional information. If the repair price exceeds the preset price limit, the prices of all items are reduced proportionally and the details are updated. Finally, all information, including the original total price, final total price, discount difference, discount, regional coefficient, fairness score, and item details, is integrated to return the complete pricing result.

[0044] In one embodiment, the initial processing file is validated based on preset rules, including: Based on the parts knowledge base, the consistency of the names of valid parts in the initial processing file is checked. Once the name consistency check passes, a price reasonableness check is performed based on the reference price range and the repair price of each valid part in the initial processing file. Once the price reasonableness check passes, a conflict check is performed on the positions of valid parts in the initial processing file based on the preset parts location rules.

[0045] Specifically, the name of each component in the initial processing file is compared with a pre-defined component knowledge base to ensure the accuracy and standardization of component names and prevent errors in large language model recognition. Only after each component name in the initial processing file successfully matches the knowledge base is the repair price of the component verified. The repair price of each component is compared with the "reference price range" in the labor hour standard library to prevent over-repair and determine whether the repair price is within a reasonable range. If the quoted price is significantly higher than the reference range, the repair price is recalculated.

[0046] When the repair price is within a reasonable range, based on the "preset parts location rules," the installation locations of the parts are checked for logical conflicts to prevent repair logic errors or over-repair. For example, if the initial processing document shows replacement of both the "left front door glass" and the "right front door glass," but the collision scene image shows only the left side is damaged; or if the initial processing document shows the installation location of a part that does not exist in a certain model, it will be judged as a "conflict." This ensures the feasibility and authenticity of the repair of the damaged vehicle and also improves the accuracy of the repair shop's damage assessment document pricing.

[0047] In one embodiment, vehicle damage-related files include relevant text and relevant images; the vehicle damage-related files are input into a trained multimodal parsing model to obtain initial recognition information, including: The multimodal parsing model includes a text recognition module, an image recognition module, and a cross-attention module. The text recognition module identifies vehicle damage based on relevant text to obtain text recognition features; the image recognition module identifies vehicle damage based on relevant images to obtain image recognition features; and the cross-attention module calculates the attention weight between the text recognition features and the image recognition features, and then performs weighted fusion of the text recognition features and the image recognition features based on the attention weight to obtain the initial recognition information.

[0048] Specifically, the multimodal parsing model includes a text recognition module, an image recognition module, and a cross-attention module. The image recognition module parses and recognizes images in vehicle damage-related documents to obtain image recognition features. The text recognition module parses and recognizes text in vehicle damage-related documents to obtain text recognition features, including a list of words extracted by OCR and bounding boxes for each word.

[0049] The cross-attention module encodes the bounding box coordinates of the text, generating corresponding spatial features. These spatial features are then superimposed on the original text recognition features to obtain new text recognition features that fuse location information. Next, using the superimposed text recognition features as the query term and the image recognition features as the key term, the cross-attention mechanism completes the association matching between the image and text, outputting the attention calculation results and weight distribution. Based on the weights, the text recognition features and image recognition features are weighted and fused, finally outputting the fused overall features, which are used as the initial recognition information.

[0050] Traditional OCR systems suffer from significantly reduced accuracy in recognizing non-standard formats, complex tables, or handwritten content, and perform poorly in processing multi-source heterogeneous documents. This application employs a multimodal parsing model as the core parser, combining visual localization and semantic understanding capabilities. It can adaptively parse arbitrary layouts without relying on fixed templates, and by introducing a cross-attention mechanism, it can accurately identify vehicle damage. Real-world testing shows that the multimodal parsing model achieves a 92.7% accuracy rate in a mixed dataset covering 127 types of non-standard documents, even in complex tables and handwritten interference scenarios, nearly three times higher than traditional OCR.

[0051] In one embodiment, the multimodal parsing model has a semantic validation rule engine module. One rule is that accessory names cannot appear in the work hours field. If the work hours field contains accessory keywords such as bumper, headlight, or glass, the content is marked for manual review, and a classification error is indicated. Another rule is that duplicate accessories are not allowed in the document. If duplicate accessory entries are detected, the content is merged or marked for review, and a duplicate entry is indicated. A third rule requires that the damaged area and repair items maintain logical consistency.

[0052] The system iterates through the three preset semantic validation rules one by one: using the judgment conditions of the current rule, it filters out all abnormal fields that trigger the rule; if there are violating fields, it generates a record containing the rule number, rule name, violating field, prompt text, and severity; if the rule requires cross-verification of text and images, it calls the visual validation tool for secondary verification; if the visual validation still fails, it is classified as an error item; if the visual validation is successful, it is classified as a warning item; for the remaining rules, the records are added to the error list or warning list according to their severity.

[0053] In one embodiment, the method for processing vehicle damage information further includes: When the valid damage information includes information that requires manual review, the system receives feedback information on the manual review, corrects the valid damage information based on the feedback information, and fine-tunes the multimodal analytical model and the large language model based on the corrected valid damage information and the corresponding vehicle damage-related documents.

[0054] Specifically, when the large language model identifies vehicle damage-related documents, if the identified parts exceed the knowledge constraints in the constraint prompt document, these parts are marked as uncertain and require manual review. When valid damage information includes information requiring manual review, feedback information from the manual review is received, and the valid damage information is corrected based on this feedback. All corrected valid damage information is collected within a preset time period. When the number of corrected valid damage information exceeds a preset threshold, it is used as incremental training data. When the number of corrected valid damage information is less than the preset threshold, this fine-tuning is skipped to avoid model overfitting or performance degradation due to insufficient data.

[0055] Obtain the currently used multimodal parsing model and large language model, and use the fine-tune-with-lora method to incrementally train both models using incremental training data. After incremental training, test the performance of the fine-tuned model on the validation set, calculate core metrics, and compare the new model's core metrics with preset thresholds. If the new model performs better, use the fine-tuned model. Through model fine-tuning, improve the model's recognition accuracy.

[0056] Furthermore, as Figure 1 In terms of specific implementation, this application provides a vehicle damage information processing device, such as... Figure 4 As shown, the device includes: The initial identification module 402 is used to obtain vehicle damage-related documents provided by the repair shop, input the vehicle damage-related documents into the trained multimodal parsing model, and obtain initial identification information, wherein the initial identification information includes the vehicle model, the name of at least one initially identified part and its corresponding working time keywords; The re-identification module 404 is used to match the vehicle model and the name of each initially identified part and its corresponding working time keywords with the preset knowledge base. Based on the name of the matched re-identified part, the reference standard working time and the reference price range, a constraint prompt file is generated. The vehicle damage-related files and the constraint prompt file are input into the trained large language model to obtain effective damage information, which includes effective parts. The solution determination module 406 is used to obtain the repair information of valid parts from the preset repair knowledge base based on the valid damage information, determine the valid repair solution according to the vehicle model and the repair information of valid parts, and generate an initial processing file based on the valid damage information and the valid repair solution. The verification module 408 is used to verify the initial processing file based on preset rules, and the initial processing file that passes the verification is used as the final processing file.

[0057] This application provides a payment receipt processing device. Compared with the prior art, it inputs vehicle damage-related documents provided by the repair shop into a trained multimodal parsing model to obtain initial identification information. The device then matches the vehicle model, the name of at least one initially identified part, and its corresponding labor time keywords from the initial identification information with a preset knowledge base. Based on the name of the matched re-identified part, reference standard labor time, and reference price range, it generates a constraint prompt file. The vehicle damage-related documents and constraint prompt file are then input into a trained large language model to obtain valid damage information. Based on this valid damage information, it retrieves repair information for valid parts from a preset repair knowledge base. Based on the vehicle model and the repair information for valid parts, it determines an effective repair plan and generates an initial processing file. The initial processing file is then validated based on preset rules, and the validated initial processing file is used as the final damage assessment. By matching the initially identified parts with the knowledge base, a constraint prompt file is generated. The large language model performs vehicle damage identification under the constraints of the constraint prompt file, without fabricating part names. This improves the accuracy and compliance of vehicle damage information processing, increases the efficiency of insurance company claims processing, and significantly improves user trust and service satisfaction.

[0058] In one embodiment, the preset knowledge base includes a parts knowledge base and a standard working hour base; the re-identification module is also used for: Obtain the names of all candidate parts that match the vehicle model from the parts knowledge base. For each initially identified part, calculate the similarity between the name of the initially identified part and the name of each candidate part. When the maximum similarity value is greater than the preset similarity threshold, use the name of the candidate part corresponding to the maximum similarity value as the name of the re-identified part. Based on the name of each re-identified part and its corresponding time keywords, the time is matched in the standard time database to obtain the reference standard time and reference price range of the re-identified part. Obtain a preset constraint template, which includes knowledge constraints, task constraints, and rule constraints; Enter the vehicle model, the name of each re-identification part, the reference standard working hours and reference price range for each re-identification part into the knowledge constraints to generate a constraint prompt file.

[0059] In one embodiment, task constraints are used to constrain the large language model to identify vehicle damage in relevant documents as a vehicle insurance damage assessment expert, and output the identified vehicle damage in a preset format; knowledge constraints are used to use the name of the re-identified parts, the reference standard working hours for each re-identified part, and the reference price range as a knowledge pool, and constrain the large language model to identify vehicle damage in relevant documents based on the knowledge pool; rule constraints are used to constrain the large language model to identify vehicle damage in relevant documents based on the knowledge constraints, and when the knowledge constraints do not support the damage assessment identification of any part, it is marked as undeterminable.

[0060] In one embodiment, the scheme determination module is further configured to: Based on the vehicle model, the global map corresponding to the vehicle model is searched in the preset vehicle map. The global map includes all part nodes and their associated work hour nodes. Based on the names of valid parts in the valid damage information, in the global graph, filter the parts nodes and associated work hour nodes related to the valid parts, add repair shop nodes to the filtered graph, and associate the repair shop nodes with the parts nodes as sub-graphs; The sub-map is input into the preset prediction model to obtain the effective repair plan for each effective part. The effective repair plan includes the best repair method and its corresponding labor hours and benchmark price. Obtain the regional price adjustment coefficient corresponding to the area where the repair shop is located, and use the product of the regional price adjustment coefficient and the benchmark price as the repair price of the effective parts.

[0061] In one embodiment, the verification module is further configured to: Based on the parts knowledge base, the consistency of the names of valid parts in the initial processing file is checked. Once the name consistency check passes, a price reasonableness check is performed based on the reference price range and the repair price of each valid part in the initial processing file. Once the price reasonableness check passes, a conflict check is performed on the positions of valid parts in the initial processing file based on the preset parts location rules.

[0062] In one embodiment, the vehicle damage-related documents include relevant text and relevant images, and the initial identification module is further used for: The multimodal parsing model includes a text recognition module, an image recognition module, and a cross-attention module. The text recognition module identifies vehicle damage based on relevant text to obtain text recognition features; the image recognition module identifies vehicle damage based on relevant images to obtain image recognition features; and the cross-attention module calculates the attention weight between the text recognition features and the image recognition features, and then performs weighted fusion of the text recognition features and the image recognition features based on the attention weight to obtain the initial recognition information.

[0063] In one embodiment, the vehicle damage information processing apparatus further includes: The fine-tuning module is used to receive feedback information from manual review when the valid damage information includes information that requires manual review. Based on the feedback information, the module corrects the valid damage information and then fine-tunes the multimodal analytical model and the large language model based on the corrected valid damage information and the corresponding vehicle damage-related files.

[0064] It should be noted that other corresponding descriptions of the functional units involved in the payment and receipt processing device provided in this application embodiment can be found by referring to... Figures 1 to 3 The corresponding descriptions in the method will not be repeated here.

[0065] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, memory, and communication interface connected via a system bus, and may also include input / output interfaces and a display device. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a payment / receipt processing method on the server side.

[0066] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain vehicle damage-related documents from the repair shop, input the vehicle damage-related documents into the trained multimodal parsing model to obtain initial identification information, which includes the vehicle model, the name of at least one initially identified part and its corresponding labor time keywords; The vehicle model and the name of each initially identified part, along with their corresponding working time keywords, are matched with a pre-set knowledge base. A constraint prompt file is generated based on the name of the re-identified part, the reference standard working time, and the reference price range. The vehicle damage-related files and the constraint prompt file are then input into the trained large language model to obtain valid damage information, which includes valid parts. Based on valid damage information, the system retrieves repair information for valid parts from a pre-set repair knowledge base. Based on the vehicle model and the repair information for valid parts, it determines an effective repair plan. Based on the valid damage information and the effective repair plan, it generates an initial processing file. The initial processing file is validated based on preset rules, and the initial processing file that passes the validation is used as the final processing file.

[0067] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain vehicle damage-related documents from the repair shop, input the vehicle damage-related documents into the trained multimodal parsing model to obtain initial identification information, which includes the vehicle model, the name of at least one initially identified part and its corresponding labor time keywords; The vehicle model and the name of each initially identified part, along with their corresponding working time keywords, are matched with a pre-set knowledge base. A constraint prompt file is generated based on the name of the re-identified part, the reference standard working time, and the reference price range. The vehicle damage-related files and the constraint prompt file are then input into the trained large language model to obtain valid damage information, which includes valid parts. Based on valid damage information, the system retrieves repair information for valid parts from a pre-set repair knowledge base. Based on the vehicle model and the repair information for valid parts, it determines an effective repair plan. Based on the valid damage information and the effective repair plan, it generates an initial processing file. The initial processing file is validated based on preset rules, and the initial processing file that passes the validation is used as the final processing file.

[0068] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0069] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0070] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0071] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0072] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing vehicle damage information, characterized in that, The method includes: Obtain vehicle damage-related documents provided by the repair shop, input the vehicle damage-related documents into the trained multimodal analytical model to obtain initial identification information, wherein the initial identification information includes the vehicle model, the name of at least one initially identified part and its corresponding work time keywords; The vehicle model and the name of each initially identified part, along with their corresponding working hour keywords, are matched with a preset knowledge base. A constraint prompt file is generated based on the name of the re-identified part, the reference standard working hours, and the reference price range. The vehicle damage-related files and the constraint prompt file are input into the trained large language model to obtain valid damage information, which includes valid parts. Based on the valid damage information, repair information for valid parts is obtained from a preset repair knowledge base. Based on the vehicle model and the repair information for the valid parts, a valid repair plan is determined. Based on the valid damage information and the valid repair plan, an initial processing file is generated. The initial processing file is verified based on preset rules, and the initial processing file that passes the verification is used as the final processing file.

2. The method for processing vehicle damage information according to claim 1, characterized in that, The preset knowledge base includes a parts knowledge base and a standard labor hour base; the process of matching the vehicle model and the name of each initially identified part and its corresponding labor hour keywords with the preset knowledge base, and generating a constraint prompt file based on the matched actual part name, reference standard labor hour, and reference price range, includes: The names of all candidate parts matching the vehicle model are obtained from the parts knowledge base. For each initially identified part, the similarity between the name of the initially identified part and the name of each candidate part is calculated. When the maximum similarity value is greater than a preset similarity threshold, the name of the candidate part corresponding to the maximum similarity value is used as the name of the re-identified part. Based on the name of each re-identified part and its corresponding time keywords, the time is matched in the standard time database to obtain the reference standard time and reference price range of the re-identified part. Obtain a preset constraint template, wherein the constraint template includes knowledge constraints, task constraints, and rule constraints; The vehicle model, the name of each re-identification part, the reference standard working hours and reference price range of each re-identification part are filled into the knowledge constraint to generate a constraint prompt file.

3. The method for processing vehicle damage information according to claim 2, characterized in that, The task constraint is used to constrain the large language model to identify the vehicle damage situation in the vehicle damage-related documents according to the role of a car insurance damage assessment expert, and to output the identified vehicle damage situation in a preset format; The knowledge constraint is used to define the name of the re-identified part, the reference standard working hours for each re-identified part, and the reference price range as a knowledge pool, constraining the large language model to identify vehicle damage based on the knowledge pool; the rule constraint is used to constrain the large language model to identify vehicle damage based on the knowledge constraint, and when the knowledge constraint does not support the damage assessment of any part, it is marked as undeterminable.

4. The method for processing vehicle damage information according to claim 1, characterized in that, The step of determining an effective repair plan based on the vehicle model and the repair information of the available parts includes: Based on the vehicle model, the global map corresponding to the vehicle model is searched in the preset vehicle map, wherein the global map includes all component nodes and associated work hour nodes; Based on the names of valid parts in the valid damage information, in the global graph, the parts nodes and associated work hour nodes related to the valid parts are filtered out, and a repair shop node is added to the filtered graph. The repair shop node is associated with the parts node as a sub-graph. The sub-map is input into a preset prediction model to obtain an effective repair plan for each effective component. The effective repair plan includes the optimal repair method and its corresponding labor hours and benchmark price. Obtain the regional price adjustment coefficient corresponding to the region where the repair shop is located, and multiply the regional price adjustment coefficient by the benchmark price as the repair price of the valid parts.

5. The method for processing vehicle damage information according to claim 2, characterized in that, The verification of the initial processed file based on preset rules includes: Based on the accessory knowledge base, the consistency of the names of valid accessories in the initial processing file is checked. Once the name consistency check passes, a price reasonableness check is performed based on the reference price range and the repair price of each valid part in the initial processing file. Once the price reasonableness check passes, a conflict check is performed on the positions of valid accessories in the initial processing file based on preset accessory position rules.

6. The method for processing vehicle damage information according to any one of claims 1-5, characterized in that, The vehicle damage-related files include relevant text and relevant images; the step of inputting the vehicle damage-related files into the trained multimodal parsing model to obtain initial recognition information includes: The multimodal parsing model includes a text recognition module, an image recognition module, and a cross-attention module. The text recognition module identifies vehicle damage based on the relevant text to obtain text recognition features. The image recognition module identifies vehicle damage based on the relevant images to obtain image recognition features. The cross-attention module calculates the attention weight between the text recognition features and the image recognition features, and performs weighted fusion of the text recognition features and the image recognition features based on the attention weight to obtain initial recognition information.

7. The method for processing vehicle damage information according to any one of claims 1-5, characterized in that, The method for processing vehicle damage information also includes: When the valid damage information includes information that requires manual review, feedback information on the manual review information is received, the valid damage information is corrected based on the feedback information, and the multimodal parsing model and the large language model are fine-tuned based on the corrected valid damage information and the corresponding vehicle damage-related files.

8. A device for processing vehicle damage information, characterized in that, The device includes: The initial identification module is used to obtain vehicle damage-related documents provided by the repair shop, input the vehicle damage-related documents into the trained multimodal parsing model, and obtain initial identification information, wherein the initial identification information includes the vehicle model, the name of at least one initially identified part and its corresponding work time keywords; The re-identification module is used to match the vehicle model and the name of each initially identified part and its corresponding working time keywords with a preset knowledge base. Based on the name of the matched re-identified part, the reference standard working time and the reference price range, a constraint prompt file is generated. The vehicle damage-related files and the constraint prompt file are input into the trained large language model to obtain effective damage information, which includes effective parts. The solution determination module is used to obtain repair information of valid parts from a preset repair knowledge base based on the valid damage information, determine a valid repair solution based on the vehicle model and the repair information of the valid parts, and generate an initial processing file based on the valid damage information and the valid repair solution. The verification module is used to verify the initial processing file based on preset rules, and the initial processing file that passes the verification is used as the final processing file.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images