Intelligent ship repair quotation method and system based on multi-modal analysis and large language model

Abstract

The invention relates to an intelligent ship repair quotation method and system based on multi-modal analysis and a large language model, and the method comprises the steps: firstly receiving a ship repair inquiry sheet file uploaded by a user, and carrying out the feature enhancement of image data through a DocEnTR model, and obtaining a binary image; automatically extracting unstructured text data, semi-structured table data and text information in the binarized image by adopting an OCR (Optical Character Recognition) technology, and performing multi-modal analysis on image features of the binarized image and the extracted text information by adopting a large language model to generate an initial data stream; synchronously generating a standard semantic tag and a non-standard content tag by adopting semantic association of the large language model, automatically generating a dynamic cue word according to the standard semantic tag, inputting the dynamic cue word into the large language model, and outputting a standardized engineering description; performing manual auditing on the non-standard content mark to correct the engineering item field so as to generate structured engineering data; and finally, calculating the total price based on the structured engineering data and the material quantity parameters, and further generating a final quotation list to complete ship repair quotation.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent information processing and automated quotation, and in particular to an intelligent ship repair quotation method and system based on multimodal analysis and a large language model. Background Art

[0002] The ship repair industry (especially large-scale ship repair companies) handles a large number of project quotation requests annually, including repair, refit, and maintenance services for both domestic and international vessels. Processing and understanding quotation sheets is crucial to ensuring smooth business operations. Currently, heavy industry companies are required to submit a massive number of quotations annually, approximately 4,000 to 5,000, covering the repair needs of numerous domestic and international vessels. This process involves detailed analysis of the RFQs, precise evaluation of materials and construction items, and the generation of the final quotation.

[0003] Currently, most heavy industry companies use traditional manual methods to prepare these quotations. Assuming a single quotation takes an average of three days to calculate, this requires 48 to 60 sales representatives, who are solely responsible for quoting. If additional ship repair bids are needed, this can only be accomplished by adding staff, which underutilizes human resources. Specifically, sales representatives must manually parse each RFQ, manually upload documents for approval, and manually review historical quotations, extracting key information such as materials, construction items, and technical parameters. This information is then used to calculate prices and compile quotations. This approach relies on individual experience and judgment, and typically takes an average of three days to complete a single quotation.

[0004] Although the above methods have met the quotation needs to a certain extent, their shortcomings are also obvious: 1) Time-consuming: Manual processing is inefficient. From reading documents, extracting information to the formation of the final quotation, the entire process takes a long time and involves a lot of copying and pasting. 2) High error rate: Due to the heavy reliance on manual operations, including copying and pasting, human errors are prone to occur, affecting the accuracy of the quotation. 3) Poor scalability: With the growth of business, if the number of ship repair bids is to be increased, it can only be achieved by adding staff, which not only increases costs but also makes it difficult to use human resources efficiently. 4) Inconvenient data management: There is a lack of unified data management and specifications. Manual review of historical quotations is not conducive to quick comparison and reference, and it is difficult to support efficient decision-making. 5) Lack of standardization: Due to the different habits of each person, the data format is not unified, which brings obstacles to subsequent data analysis.

[0005] To sum up, in order to address these problems, a smarter and more efficient solution is urgently needed to automate and intelligentize the quotation process, improve work efficiency, reduce human errors, and optimize data management. Summary of the Invention

[0006] To address the current challenges of ship repair quotation processing, such as long processing time, high error rates, poor scalability, inconvenient data management, and a lack of standardization, the present invention provides an intelligent ship repair quotation method based on multimodal analysis and a large language model. This method automatically generates historical quotations and can upload and parse files in any format. This method effectively shortens quotation preparation time, improves quotation efficiency, and significantly enhances the automation and accuracy of ship repair quotation processing. The present invention also relates to an intelligent ship repair quotation system based on multimodal analysis and a large language model.

[0007] The technical solutions of the present invention are as follows:

[0008] An intelligent ship repair quotation method based on multimodal analysis and a large language model, characterized by comprising the following steps:

[0009] The feature enhancement and initial data stream generation steps include: receiving a ship repair inquiry document uploaded by a user, the ship repair inquiry document including unstructured text data, semi-structured table data, and image data; performing feature enhancement processing on the image data using the DocEnTR model to obtain a feature-enhanced binary image; automatically extracting text information from the unstructured text data, semi-structured table data, and binary image using OCR technology; and performing multimodal analysis on the image features of the binary image and the extracted text information using a large language model to generate an initial data stream including engineering item fields and physical quantity parameter fields;

[0010] Standardized engineering description and engineering item field correction steps: Use a large language model to semantically associate the initial data stream, generating standard semantic tags and non-standard content tags; then automatically generate dynamic prompt words for domestic and / or foreign ships based on the standard semantic tags, input the dynamic prompt words into the large language model, and output standardized engineering descriptions; push the non-standard content tags to the manual review interface, and automatically correct the engineering item fields in the initial data stream based on the review results; generate structured engineering data that meets ship industry standards based on the standardized engineering description and the corrected engineering item fields;

[0011] Price calculation and quotation generation steps: Based on the structured engineering data and the price book / agreement price list, a multi-dimensional price matching algorithm that includes ship type, engineering complexity, and historical price comparison is used to generate a benchmark unit price according to the weight coefficient of each dimension. The total price is calculated based on the benchmark unit price and the physical quantity parameters in the standardized engineering description. Then, a standardized quotation that complies with industry standards and includes standardized quotation items such as standardized engineering description, quantity, benchmark unit price, and total price is automatically generated to complete the ship repair quotation.

[0012] Preferably, in the feature enhancement and initial data stream generation steps, the feature enhancement processing of the image data using the DocEnTR model specifically includes:

[0013] Hough transform is used to correct the tilt of the document edge in the image data, and then the CycleGAN adversarial network is used to remove shadows and enhance illumination on the corrected image data. The processed image data is converted into a black and white binary image to obtain a feature-enhanced binary image.

[0014] Preferably, in the feature enhancement and initial data stream generation steps, the Apache Jena rule engine is further used to perform unit consistency detection on the quantity parameter field, and when a conflict in the unit of the quantity parameter field is detected, it is automatically converted to a standard unit.

[0015] Preferably, the large language model used is the Qwen2.5.32B language model.

[0016] Preferably, in the step of standardizing the project description and correcting the project item fields, a large language model is used to semantically associate the initial data stream, and generating standard semantic tags and non-standard content tags respectively, specifically including:

[0017] The standard terms corresponding to the engineering item fields in the initial data stream are queried through the term library, and the engineering item fields in the initial data stream are updated with the standard terms to generate standard semantic tags containing construction types and technical specifications; then the fuzzy descriptions in the initial data stream are output as non-standard content tags.

[0018] Preferably, in the price calculation and quotation generation steps, an additional rate is determined based on the additional terms in the standardized project description, and the additional rate is multiplied by the base unit price to generate a final unit price. The total price is calculated based on the final unit price and the quantity parameters in the standardized project description, and then a standardized quotation that complies with industry standards and includes standardized quotation items such as the standardized project description, quantity, base unit price, final unit price and total price is automatically generated to complete the ship repair quotation.

[0019] An intelligent ship repair quotation system based on multimodal analysis and a large language model is characterized by comprising a feature enhancement and initial data stream generation module, a standardized project description and project item field correction module, and a price calculation and quotation generation module connected in sequence.

[0020] The feature enhancement and initial data stream generation module receives a ship repair quotation form file uploaded by a user, the ship repair quotation form file including unstructured text data, semi-structured table data, and image data; performs feature enhancement processing on the image data using the DocEnTR model to obtain a feature-enhanced binary image; and automatically extracts text information from the unstructured text data, semi-structured table data, and binary image using OCR technology. A large language model is then used to perform multimodal analysis on the image features of the binary image and the extracted text information to generate an initial data stream including engineering item fields and physical quantity parameter fields.

[0021] The standardized engineering description and engineering item field correction module uses a large language model to semantically associate the initial data stream, generating standard semantic tags and non-standard content tags. Dynamic prompt words for Chinese and / or foreign vessels are automatically generated based on the standard semantic tags, and the dynamic prompt words are input into the large language model to output standardized engineering descriptions. The non-standard content tags are pushed to a manual review interface, and the engineering item fields in the initial data stream are automatically corrected based on the review results. Structured engineering data that complies with shipbuilding industry standards is generated based on the standardized engineering descriptions and the corrected engineering item fields.

[0022] The price calculation and quotation generation module, based on the structured engineering data linked to the price book / agreement price list, generates a benchmark unit price according to the weight coefficient of each dimension through a multi-dimensional price matching algorithm that includes the ship type dimension, the engineering complexity dimension, and the historical price comparison dimension. The module also calculates the total price based on the benchmark unit price and the physical quantity parameters in the standardized engineering description, and then generates a standardized quotation sheet that complies with industry standards and includes standardized quotation items such as the standardized engineering description, quantity, benchmark unit price, and total price, thereby completing the ship repair quotation.

[0023] Preferably, in the feature enhancement and initial data stream generation module, the feature enhancement processing of the image data using the DocEnTR model specifically includes:

[0024] Hough transform is used to correct the tilt of the document edge in the image data, and then the CycleGAN adversarial network is used to remove shadows and enhance illumination on the corrected image data. The processed image data is converted into a black and white binary image to obtain a feature-enhanced binary image.

[0025] Preferably, the feature enhancement and initial data stream generation module further uses the Apache Jena rule engine to perform unit consistency detection on the quantity parameter field, and automatically converts the unit into a standard unit when a conflict in the unit of the quantity parameter field is detected.

[0026] Preferably, in the standardized project description and project item field correction module, a large language model is used to parse the initial data stream, and the generation of standard semantic tags and non-standard content tags respectively includes:

[0027] The standard terms corresponding to the engineering item fields in the initial data stream are retrieved from the term library, and the engineering item fields in the initial data stream are updated with the standard terms to generate standard semantic tags containing construction types and technical specifications; the fuzzy descriptions in the initial data stream are then output as non-standard content tags;

[0028] And / or, the price calculation and quotation generation module further determines an additional rate based on the additional terms in the standardized project description, multiplies the additional rate by the base unit price to generate a final unit price, and calculates the total price based on the final unit price and the quantity parameters in the standardized project description, thereby automatically generating a standardized quotation that complies with industry standards and includes standardized quotation items such as the standardized project description, quantity, base unit price, final unit price, and total price, thereby completing the ship repair quotation.

[0029] The technical effects of the present invention are as follows:

[0030] The present invention provides an intelligent ship repair quotation method based on multimodal analysis and a large language model. First, a ship repair inquiry form file uploaded by a user is received. The ship repair inquiry form file includes unstructured text data, semi-structured table data, and image data. Since poor image data quality may lead to subsequent OCR recognition errors, thereby affecting the extraction of engineering item fields and the accuracy of the final quotation, the DocEnTR model is used to perform feature enhancement processing on the image data to obtain a binary image after feature enhancement. This effectively improves image clarity, eliminates interference information, and better separates text from background, greatly improving the subsequent OCR recognition accuracy, avoiding OCR misreading of irrelevant content, ensuring the quality of system input data, and improving the stability and intelligence level of the overall process. OCR technology is then used to extract textual information from unstructured text data, semi-structured tabular data, and binary images. A large language model (LLM) is then used to perform multimodal analysis on the image features of the binary images and the extracted textual information to generate the initial data stream. This automated text extraction reduces the error rate in handwritten engineering order recognition from 15% for manual entry to below 2%, effectively eliminating manual entry errors and significantly reducing the workload for manual proofreading. Furthermore, the integration of multi-dimensional data, including images, text, and tables, enhances engineering order parsing capabilities. Automatically merging multi-source data reduces file parsing time from four hours for traditional manual processing to 10 minutes, significantly improving processing efficiency. The large language model is then used to semantically correlate the initial data stream, generating standard semantic tags and non-standard content tags. Standard terms such as "steel plate replacement" are automatically identified and mapped to the standard semantic tags, reducing the need for manual intervention. Non-standard content tagging, which marks ambiguous content (such as "needs processing"), maintains business flexibility and reduces manual review workload by 70%. Then, dynamic prompt words for domestic and / or foreign ships are automatically generated based on standard semantic tags, and the dynamic prompt words are input into the large language model to output standardized engineering descriptions; non-standard content tags are pushed to the manual review interface to trigger manual review, and the review results are received and the engineering item fields in the initial data stream are automatically corrected based on the review results; structured engineering data that meets the ship industry standards is generated based on the standardized engineering description and the corrected engineering item fields. The "dynamic prompt words + model feedback" mechanism is used to further optimize the semantic parsing accuracy and improve the intelligence level of the system. At the same time, combined with the manual review link, the accuracy and compliance of key fields are ensured, taking into account both automation efficiency and business specification requirements. The structured engineering data finally generated can be directly used in subsequent processes such as price calculation and cost analysis, thereby improving the response speed and reliability of the overall quotation system.Finally, based on the structured engineering data, the price book / agreement price list is linked to the price list. A multi-dimensional price matching algorithm, encompassing ship type, project complexity, and historical price comparison, generates a benchmark unit price based on the weighting coefficients of each dimension. The total price is then calculated based on the benchmark unit price and the quantity parameters in the standardized engineering description. This automatically generates standardized quotation items, including the standardized engineering description, quantity, benchmark unit price, and total price. This results in a standardized quotation sheet that complies with industry standards and includes standardized quotation items, completing the ship repair quotation. This creates an end-to-end closed-loop process from semantic parsing to price generation. Intelligent matching with the agreement price list allows for rapid identification of corresponding items, eliminating manual search errors or omissions. Automatic pricing based on standardized engineering descriptions and quantity parameters ensures consistency and transparency in quotation logic, improving quotation accuracy and professionalism. The generated quotation sheet is clearly structured and complete, compliant with industry standards, facilitating customer understanding and facilitating internal approval processes. Furthermore, the system supports historical quotation review and data analysis, providing strong support for subsequent business negotiations, contract management, and cost control.

[0031] The present invention integrates a number of cutting-edge technologies such as image enhancement, OCR recognition, natural language processing, semantic understanding and intelligent pricing, and builds a full-process automated solution from original document parsing to structured engineering data generation to intelligent quotation output. Compared with the traditional method that relies on manual input and judgment, it has higher processing efficiency, stronger fault tolerance and better cost control capabilities, and can significantly improve the quotation response speed and service quality of ship repair companies. It has good promotion value and engineering application prospects. The present invention is not limited to document formats. WORD, PDF, and EXCEL can all be uploaded for file parsing, and the original manual copy and paste work is transformed into one-click document upload and parsing; the one-click upload and parsing work saves the business representative a lot of mechanized and tedious work; the time to prepare a quotation is shortened from 3-5 days to 1-2 days, and more quotations can be prepared in the same time, effectively improving the quotation efficiency. Moreover, all quotation data of the present invention are integrated into the system, which is convenient for the subsequent management and analysis of quotation sheets. Since traditional quotation data is not uniformly managed, subsequent analysis is relatively cumbersome and it is difficult to conduct efficient data analysis. The present invention breaks the data island and can subsequently connect to BI tools through standardized data interfaces to automatically generate multi-dimensional analysis reports (core indicators such as quotation success rate). It can efficiently process various types of data such as unstructured text, tables and images, solving the problems of low efficiency and easy errors in traditional manual data entry, and significantly improving the automation level and accuracy of ship repair quotation work. The DocEnTR model is used to enhance the features of images and extract text information in combination with OCR technology, which improves the recognition accuracy in complex image scenes; the Qwen2.5.32B language model is used to achieve standardization of engineering descriptions and automatic field correction, effectively unifying the expression of industry terms and enhancing the system's understanding and adaptability to non-standard expressions; finally, based on structured engineering data, the agreement price list is automatically matched and the pricing logic is completed, realizing the standardization, rapidity and traceability of the quotation process.

[0032] Furthermore, the Hough transform is used to correct the tilt of the document edges in the image data, eliminating the document tilt caused by improper shooting angles. The CycleGAN adversarial network is then used to remove shadows and enhance the illumination of the corrected image data, effectively improving the image quality degradation caused by uneven ambient lighting or occlusion. The processed image data is then converted into a black and white binary image to obtain a feature-enhanced binary image, which not only reduces redundant information interference but also improves OCR recognition efficiency and the overall performance of the system. Through multi-stage collaborative processing, the clarity and structural integrity of the original image are significantly improved, and the recognition accuracy of subsequent OCR technology for text areas is enhanced, especially in low-quality images, complex backgrounds, or poorly illuminated images.

[0033] Furthermore, the Apache Jena rules engine is used to perform unit consistency checks on quantity parameter fields. When conflicts are detected, the system automatically converts them to standard units. This intelligently identifies and corrects unit inconsistencies across different project items, thus avoiding price deviations and project errors caused by incorrect units. Compared to traditional manual verification methods, this significantly improves data processing accuracy and efficiency, while also enhancing the system's adaptability to non-standardized input data.

[0034] Furthermore, a large language model is used to perform semantic association on the initial data stream, and standard semantic tags and non-standard content tags are generated respectively. Specifically, the following steps are taken: standard terms corresponding to the engineering item fields in the initial data stream are queried through the term library, and the engineering item fields in the initial data stream are updated with standard terms to generate standard semantic tags containing construction types and technical specifications; the fuzzy descriptions in the initial data stream are then output as non-standard content tags. With the help of the powerful context understanding and term mapping capabilities of the Qwen2.5.32B language model, combined with the preset professional term library in the field of ship maintenance, automatic recognition and standardized conversion of non-standard engineering descriptions are realized. Through the replacement of standard terms, the engineering expression caliber is effectively unified, and the quality and availability of subsequent structured data are improved. It is particularly suitable for processing ship repair inquiry data from different users and sources in different formats.

[0035] Furthermore, an additional rate is determined based on the additional terms in the standardized project description and multiplied by the base unit price to generate the final unit price. By analyzing the additional terms in the standardized project description (such as special construction environments, process requirements, and material substitutions), the system automatically identifies key factors affecting cost and dynamically calculates the additional rate based on a pre-set rate rule library. Compared to traditional manual judgment methods, this significantly improves the accuracy and consistency of quotation calculations, avoiding pricing errors caused by human oversight or misunderstandings.

[0036] The present invention also relates to an intelligent ship repair quotation system based on multimodal analysis and a large language model. This system corresponds to the above-mentioned intelligent ship repair quotation method based on multimodal analysis and a large language model, and can be understood as a system for implementing the above-mentioned intelligent ship repair quotation method based on multimodal analysis and a large language model. The system includes a feature enhancement and initial data stream generation module, a standardized project description and project item field correction module, and a price calculation and quotation generation module connected in sequence. The modules work together and, by integrating key technologies such as image processing, natural language understanding, and structured data generation, construct a complete technical process from original inquiry document input to intelligent quotation output. The system can efficiently process various types of data such as unstructured text, tables, and images, solves the problems of low efficiency and error-proneness of traditional manual data entry, and significantly improves the automation level and accuracy of ship repair quotation work. The DocEnTR model is used to enhance image features and combine with OCR technology to extract text information, improving recognition accuracy in complex image scenarios. The Qwen2.5.32B language model is used to standardize and correct fields in engineering descriptions, effectively unifying industry terminology and enhancing the system's ability to understand and adapt to non-standard expressions. Finally, the agreement price list is automatically matched based on structured engineering data and the pricing logic is completed, achieving standardization, acceleration and traceability of the quotation process.

[0037] In addition, the system has a built-in and complete data permission management mechanism, which is helpful for dynamic personnel adjustments. Unified management can be achieved by simply adjusting the functions and data permission configurations of the personnel in the system. It adopts a three-level management and control system of "organization-role-data" to support fine-grained permission policy configuration, such as field-level data visibility control, approval flow operation permission allocation, etc., to ensure that personnel in different positions can only access data content within their scope of responsibility. At the same time, the system has a built-in dynamic adaptation engine for the organizational structure. When personnel positions change, their permission configurations can be automatically and synchronously updated to ensure data security and compliance in business scenarios with frequent personnel turnover. The present invention not only improves the quotation response speed and service quality of ship repair companies, but also provides a solid data foundation for subsequent contract management, cost analysis, permission control and decision support. It has significant engineering application value and commercial promotion potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the intelligent ship repair quotation method based on multimodal analysis and large language model of the present invention.

[0039] Figure 2 This is a schematic diagram of the intelligent ship repair quotation method based on multimodal analysis and a large language model of the present invention. DETAILED DESCRIPTION

[0040] The present invention will be described below with reference to the accompanying drawings.

[0041] The present invention relates to an intelligent ship repair quotation method based on multimodal analysis and large language model. The flow chart of the method is as follows: Figure 1 As shown, the following steps are included in sequence:

[0042] 1. Feature enhancement and initial data stream generation steps: receiving a ship repair inquiry document uploaded by a user, the ship repair inquiry document including unstructured text data, semi-structured table data, and image data; using the DocEnTR model to perform feature enhancement processing on the image data to obtain a feature-enhanced binary image; and using OCR technology to automatically extract text information from the unstructured text data, semi-structured table data, and binary image. Then, using a large language model, multimodal analysis is performed on the image features of the binary image and the extracted text information to generate an initial data stream; the initial data stream includes engineering item fields and physical quantity parameter fields.

[0043] Specifically, if Figure 2 As shown, the system first creates a quotation number CIIP-2023-001 (new quotation), and the user logs in to the system through the API and uploads the ship repair inquiry file (Excel / Word / PDF), that is, uploads the original engineering order, receives the ship repair inquiry file uploaded by the user, and confirms the file type (Excel / Word / PDF), distinguishes whether it is an extractable text type document or a scanned document, and applies different parsing strategies to different types to extract page-level text line information and image area information from each page of the document, laying the foundation for subsequent structured recognition; and automatically detects the header, footer information and repeated page elements at the bottom, and applies reasonable screening strategies to remove repeated page elements; in addition, the reading order is automatically rearranged according to the document layout to ensure the correctness of subsequent segmentation and extraction logic; and the distribution of each component in the document is judged, including paragraphs, titles, pictures, tables, headers, footers and other positioning information.

[0044] The present invention can achieve single sign-on with the original system through API, synchronously obtain basic data such as ships and shipowners, and ensure information consistency. It supports uploading files in formats such as Excel, Word, and PDF, and is compatible with diverse input requirements.

[0045] The ship repair inquiry document includes unstructured text data (plain text paragraphs in PDF / Word, which can be scanned copies, such as the port side steel plate corrosion needs to be treated), semi-structured table data (a bill of materials in an Excel spreadsheet, such as model: T11 steel plate, quantity 20 square meters), and image data (a scan of the steel plate corrosion area in PDF). The DocEnTR model is then used to perform feature enhancement processing on the image data, such as denoising, edge enhancement, and binarization conversion. That is, the Hough transform is used to correct the tilt of the document edge in the image data (such as automatically correcting it if the drawing is detected to be rotated 15 degrees), and the CycleGAN adversarial network is then used to remove shadows from the corrected image data. and illumination enhancement processing, and converting the processed image data into a black and white binary image to obtain a clear binary image after feature enhancement (black and white contrast enhancement, such as text is black and background is white); and using multi-language, multi-specification supported OCR technology to automatically extract unstructured text data, semi-structured table data and text information in the binary image (such as the port T11 steel plate corrosion area, about 2.5m×8m, which needs to be processed), that is, using OCR technology to perform text recognition and table recognition to extract plain text content in unstructured text data, cell text in semi-structured table data, and handwritten / printed text in the binary image, and using the large language model LargeLanguage A multimodal model (LLM) is used (preferably Deepseek-V3 or Qwen 2.5.32B) to analyze the image features and extracted text information of the binary image. This model integrates multidimensional data such as images and text to enhance the parsing capability of engineering orders and generate an initial data stream. The initial data stream includes engineering item fields and physical quantity parameter fields. For example, the engineering item field includes "corroded area of ​​the port side T11 steel plate: 2.5 m × 8 m" requiring processing; the physical quantity parameter field includes {"Quantity": "20 m2"}.

[0046] The present invention adopts an OCR engine for automatic document recognition, and can ensure recognition accuracy through engineering processing such as layout correction and text enhancement; deploys a large language model (multimodal large language model) to perform structured analysis of engineering orders, accurately understands engineering content (such as T11 / T13 / T14 models, quantities, etc.), and extracts key fields (engineering quantities, technical parameters), namely engineering item fields and quantity parameter fields; establishes semantic verification to ensure the standardized conversion of dimensional units and professional terms.

[0047] Furthermore, the Apache Jena rule engine can be used to perform unit consistency checks on quantity parameter fields. When conflicts are detected in the units of a quantity parameter field, they are automatically converted to standard units. This can intelligently identify and correct unit inconsistencies across different project items, thereby avoiding quotation deviations or project errors caused by unit errors.

[0048] 2. Standardized engineering description and engineering item field correction steps: Use a large language model to perform semantic association on the initial data stream, and generate standard semantic tags and non-standard content tags respectively; then automatically generate dynamic prompt words for domestic and / or foreign ships based on the standard semantic tags, input the dynamic prompt words into the large language model, and output standardized engineering descriptions; push the non-standard content tags to the manual review interface, and automatically correct the engineering item fields in the initial data stream based on the review results; generate structured engineering data that meets the ship industry standards based on the standardized engineering description and the corrected engineering item fields.

[0049] Specifically, if Figure 2 As shown, a large language model (preferably the Qwen 2.5.32B model) is first used to perform semantic association (or semantic understanding) on ​​the initial data stream. Standard terms corresponding to the engineering item fields in the initial data stream are retrieved from the term library. These fields are then updated with the standard terms to generate standard semantic tags containing the construction type and technical specifications. Fuzzy descriptions in the initial data stream are then output as non-standard content tags, specifically content that cannot be automatically standardized. For example, if the large language model identifies "T11 steel plate" as a standard engineering item (with the prompt "engineering"), a standard semantic tag is generated, such as {"construction type": "steel plate replacement", "technical specification": "ABS2023"}. Dynamic prompt words are automatically generated based on standard semantic tags (such as "Generate a quotation for steel plate replacement with explosion-proof requirements, quoting the VIP negotiated price"). This can include dynamic prompt words for domestic ships and foreign ships. The quotation content can be generated on the T10AI intelligent body (Qingzhou platform), and the dynamic prompt words are input into a large language model (such as the Qwen2.5.32B language model) to output a standardized engineering description (such as "Replacement of the port T11 steel plate (20 m2) requires explosion-proofing"). In addition, the large language model marks the "needs to be processed" in the initial data stream as a fuzzy description and recommends replacing it with "partial replacement", that is, the non-standard content is marked as: {"original description": "needs to be processed", "recommended replacement": "partial replacement"}. The non-standard content tags are then manually reviewed and a decision is made as to whether to adopt the suggestions (i.e., accept or reject). If not, the original data and annotations are retained. If yes, the engineering item fields in the initial data stream are automatically corrected based on the suggestions in the non-standard content. Finally, structured engineering data that complies with shipbuilding industry standards is generated based on the standardized engineering description and the corrected engineering item fields, namely {"engineering item": "Partial replacement of T11 steel plate, explosion-proof required"; "physical quantity parameters": {"quantity": "20 m2", "location": "port side"}.

[0050] The present invention builds a dynamic prompt framework based on prompt word engineering, performs context understanding through a large language model, and intelligently converts engineering requirements into standardized quotation items.

[0051] 3. Price calculation and quotation generation steps: Link the structured engineering data with the pre-established price book / agreement price list for matching, and generate the unit price information of the engineering item field in the structured engineering data according to the weight coefficient of each dimension through a multi-dimensional price matching algorithm including the ship type dimension, engineering complexity dimension and historical price comparison dimension; and calculate the total price based on the unit price information and the physical quantity parameters in the structured engineering data, and then automatically generate a quotation containing standardized quotation items. The standardized quotation items include standardized engineering description, quantity, base unit price and total price, that is, the final return quotation number, original engineering content, quotation content, unit, price, etc., based on the standardized quotation items, the quotation is generated. A standardized quotation sheet that complies with industry standards includes: a detailed list: a collection of standardized quotation items classified by project category (such as hull engineering, marine engineering), each item containing fields such as quotation number, original project content, standardized project description, unit, base unit price, quantity, and total price; cost structure: a breakdown of the cost of each quotation item (such as labor costs accounting for 40%, material costs accounting for 50%, and management costs accounting for 10%); floating description: the basis for price calculation (such as "this unit price is based on the agreed price list YZ-2025, ship type coefficient 1.2") and the comparison results with historical prices (such as "the price of the same type of project fluctuates by +5% compared with the previous quarter"). The format of the generated standardized quotation complies with the "Ship Repair Price Compilation Specification" (CB / T3837) or the industry standard specified by the customer, supports multiple format exports such as PDF / Excel, and ensures legal effectiveness through digital signatures; supports API docking with the company's existing system to achieve two-way synchronization of quotation data, and supports user confirmation and manual modification. The quotation is output in a standardized format and records user modification behavior for iterating prompt word rules to complete the quotation for smart ship repair.

[0052] Furthermore, an additional rate can be determined based on the additional terms in the standardized project description, and the additional rate can be multiplied by the base unit price to generate a final unit price. At this time, the total price is calculated based on the final unit price and the quantity parameters in the standardized project description, and then a standardized quotation sheet that complies with industry standards and includes standardized quotation items such as the standardized project description, quantity, base unit price, final unit price and total price is automatically generated to complete the intelligent ship repair quotation.

[0053] The intelligent ship repair quotation method based on multimodal analysis and large language model of the present invention has the following working principle: after the user logs in to the system through API and uploads the inquiry engineering form (Excel / Word / PDF), the present invention calls DocEnTR model, OCR engine, and large language model to perform image enhancement processing, document automatic recognition and multimodal analysis on the file, and extracts the initial data stream containing engineering item fields and material quantity parameter fields; the initial data stream is semantically associated with the large language model to generate standard semantic tags and non-standard content tags, and then a standardized engineering description is output based on dynamic prompt words, implicit requirements (such as "replacement") are identified, and associated with the ships and agreement price list in the database, and the engineering material quantity (such as quantity, model) is automatically matched with the price item to generate a preliminary price list; it can be as follows Figure 2 As shown, through collaborative processing within the AI ​​intelligent agent architecture, prompt word engineering: through preset rules—standardized quotation items—constrain the standardization of generated content; workflow scheduling: serial analysis → quantity matching → quotation generation → price calculation → output node, ensuring end-to-end processing efficiency and triggering manual intervention in the event of anomalies (such as alarms for missing items in the price list). The large language model generates a natural language description based on the parsing results, while embedding the calculated negotiated price data. Through a multi-dimensional price matching algorithm that includes ship type dimensions, engineering complexity dimensions, and historical price comparison dimensions, the quantity parameters are weightedly matched with the negotiated price list to generate a benchmark unit price; the total price is then calculated based on the benchmark unit price × quantity × floating coefficient (such as ship type coefficient, urgency coefficient), generating the final standardized quotation and outputting it for feedback optimization.

[0054] The present invention also relates to an intelligent ship repair quotation system based on multimodal analysis and a large language model. This system corresponds to the intelligent ship repair quotation method based on multimodal analysis and a large language model, and can be understood as a system for implementing the above method. The system includes a feature enhancement and initial data stream generation module, a standardized project description and project item field correction module, and a price calculation and quotation generation module connected in sequence. Specifically,

[0055] The feature enhancement and initial data stream generation module receives a ship repair inquiry form file uploaded by a user, the ship repair inquiry form file including unstructured text data, semi-structured table data, and image data; performs feature enhancement processing on the image data using the DocEnTR model to obtain a feature-enhanced binary image; and automatically extracts text information from the unstructured text data, semi-structured table data, and binary image using OCR technology, and then performs multimodal analysis on the image features of the binary image and the extracted text information using a large language model to generate an initial data stream; the initial data stream includes a project item field and a physical quantity parameter field;

[0056] The standardized engineering description and engineering item field correction module uses a large language model to semantically associate the initial data stream, generating standard semantic tags and non-standard content tags. Dynamic prompt words for Chinese and / or foreign vessels are automatically generated based on the standard semantic tags, and the dynamic prompt words are input into the large language model to output standardized engineering descriptions. The non-standard content tags are pushed to a manual review interface, and the engineering item fields in the initial data stream are automatically corrected based on the review results. Structured engineering data that complies with shipbuilding industry standards is generated based on the standardized engineering descriptions and the corrected engineering item fields.

[0057] The price calculation and quotation generation module, based on the structured engineering data linked to the price book / agreement price list, generates a benchmark unit price according to the weight coefficient of each dimension through a multi-dimensional price matching algorithm that includes the ship type dimension, the engineering complexity dimension, and the historical price comparison dimension. The module also calculates the total price based on the benchmark unit price and the physical quantity parameters in the standardized engineering description, and then generates a standardized quotation sheet that complies with industry standards and includes standardized quotation items such as the standardized engineering description, quantity, benchmark unit price, and total price, thereby completing the ship repair quotation.

[0058] Preferably, in the feature enhancement and initial data stream generation module, the feature enhancement processing of the image data using the DocEnTR model specifically includes:

[0059] Hough transform is used to correct the tilt of the document edge in the image data, and then the CycleGAN adversarial network is used to remove shadows and enhance illumination on the corrected image data. The processed image data is converted into a black and white binary image to obtain a feature-enhanced binary image.

[0060] Preferably, in the feature enhancement and initial data stream generation module, the Apache Jena rule engine is also used to perform unit consistency detection on the physical quantity parameter field, and when a conflict in the unit of the physical quantity parameter field is detected, it is automatically converted to a standard unit.

[0061] Preferably, in the standardized project description and project item field correction module, a large language model is used to parse the initial data stream to generate standard semantic tags and non-standard content tags, including:

[0062] The standard terms corresponding to the engineering item fields in the initial data stream are queried through the term library, and the engineering item fields in the initial data stream are updated with the standard terms to generate standard semantic tags containing construction types and technical specifications; then the fuzzy descriptions in the initial data stream are output as non-standard content tags.

[0063] Preferably, in the price calculation and quotation generation module, an additional rate is determined based on the additional terms in the standardized project description, and the additional rate is multiplied by the base unit price to generate a final unit price. The total price is calculated based on the final unit price and the quantity parameters in the standardized project description, and then a standardized quotation that complies with industry standards and includes standardized quotation items such as the standardized project description, quantity, base unit price, final unit price and total price is automatically generated to complete the ship repair quotation.

[0064] This invention provides an objective and scientific intelligent ship repair quotation method and system based on multimodal analysis and a large language model. By integrating key technologies such as image processing, natural language understanding, and structured data generation, it constructs a complete technical process from the input of the original inquiry document to the output of the intelligent quotation. It can efficiently process various types of data such as unstructured text, tables, and images, solving the problems of low efficiency and error-proneness of traditional manual data entry, and significantly improving the automation level and accuracy of ship repair quotation work. The DocEnTR model is used to enhance the features of images and combine OCR technology to extract text information, improving recognition accuracy in complex image scenarios. The Qwen2.5.32B language model is used to standardize and correct fields in engineering descriptions, effectively unifying industry terminology and enhancing the system's understanding and adaptability to non-standard expressions. Finally, based on structured engineering data, the system automatically matches the agreement price list and completes the pricing logic, achieving standardization, acceleration, and traceability of the quotation process.

[0065] It should be noted that the specific embodiments described above can enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although this specification has described the present invention in detail with reference to the drawings and embodiments, those skilled in the art should understand that the present invention can still be modified or replaced with equivalents. In short, all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be included in the scope of protection of the patent for the present invention.

Claims

1. An intelligent ship repair quotation method based on multimodal analysis and large language model, characterized by: The following steps are involved: The feature enhancement and initial data stream generation steps include: receiving a ship repair inquiry document uploaded by a user, the ship repair inquiry document including unstructured text data, semi-structured table data, and image data; performing feature enhancement processing on the image data using the DocEnTR model to obtain a feature-enhanced binary image; automatically extracting text information from the unstructured text data, semi-structured table data, and binary image using OCR technology; and performing multimodal analysis on the image features of the binary image and the extracted text information using a large language model to generate an initial data stream including engineering item fields and physical quantity parameter fields; Standardized project description and project item field correction steps: Use a large language model to perform semantic association on the initial data stream, generate standard semantic tags and non-standard content tags respectively; then automatically generate dynamic prompt words for domestic and / or foreign ships based on the standard semantic tags, input the dynamic prompt words into the large language model, and output a standardized project description; push the non-standard content tags to the manual review interface, and automatically correct the project item fields in the initial data stream based on the review results; Generate structured engineering data that complies with shipbuilding industry standards based on standardized engineering descriptions and revised engineering item fields; Price calculation and quotation generation steps: Based on the structured engineering data and the price book / agreement price list, a multi-dimensional price matching algorithm that includes ship type, engineering complexity, and historical price comparison is used to generate a benchmark unit price according to the weight coefficient of each dimension. The total price is calculated based on the benchmark unit price and the physical quantity parameters in the standardized engineering description. Then, a standardized quotation that complies with industry standards and includes standardized quotation items such as standardized engineering description, quantity, benchmark unit price, and total price is automatically generated to complete the ship repair quotation.

2. The intelligent ship repair quotation method based on multimodal analysis and large language model according to claim 1 is characterized in that: In the feature enhancement and initial data stream generation steps, the feature enhancement processing of the image data using the DocEnTR model specifically includes: Hough transform is used to correct the tilt of the document edge in the image data, and then the CycleGAN adversarial network is used to remove shadows and enhance illumination on the corrected image data. The processed image data is converted into a black and white binary image to obtain a feature-enhanced binary image.

3. The intelligent ship repair quotation method based on multimodal analysis and large language model according to claim 1 is characterized in that: In the feature enhancement and initial data stream generation steps, the Apache Jena rule engine is also used to perform unit consistency detection on the quantity parameter field. When a conflict in the unit of the quantity parameter field is detected, it is automatically converted to the standard unit.

4. The intelligent ship repair quotation method based on multimodal analysis and large language model according to any one of claims 1 to 3, characterized in that: The large language model used is the Qwen2.5.32B language model.

5. The intelligent ship repair quotation method based on multimodal analysis and large language model according to claim 4 is characterized in that: In the standardized project description and project item field correction steps, a large language model is used to semantically associate the initial data stream to generate standard semantic tags and non-standard content tags, respectively. Specifically, the steps include: The standard terms corresponding to the engineering item fields in the initial data stream are queried through the term library, and the engineering item fields in the initial data stream are updated with the standard terms to generate standard semantic tags containing construction types and technical specifications; then the fuzzy descriptions in the initial data stream are output as non-standard content tags.

6. The intelligent ship repair quotation method based on multimodal analysis and large language model according to claim 1 is characterized in that: In the price calculation and quotation generation steps, an additional rate is determined based on the additional terms in the standardized project description, and the additional rate is multiplied by the base unit price to generate a final unit price. The total price is calculated based on the final unit price and the quantity parameters in the standardized project description, and then a standardized quotation that complies with industry standards and includes standardized quotation items such as the standardized project description, quantity, base unit price, final unit price, and total price is automatically generated to complete the ship repair quotation.

7. An intelligent ship repair quotation system based on multimodal analysis and large language model, characterized by: It includes the sequentially connected feature enhancement and initial data stream generation module, the standardized project description and project item field correction module, and the price calculation and quotation generation module. The feature enhancement and initial data stream generation module receives a ship repair quotation form file uploaded by a user, the ship repair quotation form file including unstructured text data, semi-structured table data, and image data; performs feature enhancement processing on the image data using the DocEnTR model to obtain a feature-enhanced binary image; and automatically extracts text information from the unstructured text data, semi-structured table data, and binary image using OCR technology. A large language model is then used to perform multimodal analysis on the image features of the binary image and the extracted text information to generate an initial data stream including engineering item fields and physical quantity parameter fields. The standardized project description and project item field correction module uses a large language model to perform semantic association on the initial data stream, generating standard semantic tags and non-standard content tags. It then automatically generates dynamic prompt words for Chinese and / or foreign vessels based on the standard semantic tags, inputs the dynamic prompt words into the large language model, and outputs a standardized project description. It then pushes the non-standard content tags to a manual review interface and automatically corrects the project item fields in the initial data stream based on the review results. Generate structured engineering data that complies with shipbuilding industry standards based on standardized engineering descriptions and revised engineering item fields; The price calculation and quotation generation module, based on the structured engineering data linked to the price book / agreement price list, generates a benchmark unit price according to the weight coefficient of each dimension through a multi-dimensional price matching algorithm that includes the ship type dimension, the engineering complexity dimension, and the historical price comparison dimension. The module also calculates the total price based on the benchmark unit price and the physical quantity parameters in the standardized engineering description, and then generates a standardized quotation sheet that complies with industry standards and includes standardized quotation items such as the standardized engineering description, quantity, benchmark unit price, and total price, thereby completing the ship repair quotation.

8. The intelligent ship repair quotation system based on multimodal analysis and large language model according to claim 7 is characterized in that: In the feature enhancement and initial data stream generation module, the feature enhancement processing of the image data using the DocEnTR model specifically includes: Hough transform is used to correct the tilt of the document edge in the image data, and then the CycleGAN adversarial network is used to remove shadows and enhance illumination on the corrected image data. The processed image data is converted into a black and white binary image to obtain a feature-enhanced binary image.

9. The intelligent ship repair quotation system based on multimodal analysis and large language model according to claim 7 is characterized in that: In the feature enhancement and initial data stream generation module, the Apache Jena rule engine is also used to perform unit consistency detection on the physical quantity parameter field. When a conflict in the unit of the physical quantity parameter field is detected, it is automatically converted to the standard unit.

10. The intelligent ship repair quotation system based on multimodal analysis and large language model according to any one of claims 7 to 9, characterized in that: In the standardized project description and project item field correction module, a large language model is used to parse the initial data stream to generate standard semantic tags and non-standard content tags, including: The standard terms corresponding to the engineering item fields in the initial data stream are retrieved from the term library, and the engineering item fields in the initial data stream are updated with the standard terms to generate standard semantic tags containing construction types and technical specifications; the fuzzy descriptions in the initial data stream are then output as non-standard content tags; And / or, the price calculation and quotation generation module further determines an additional rate based on the additional terms in the standardized project description, multiplies the additional rate by the base unit price to generate a final unit price, and calculates the total price based on the final unit price and the quantity parameters in the standardized project description, thereby automatically generating a standardized quotation that complies with industry standards and includes standardized quotation items such as the standardized project description, quantity, base unit price, final unit price, and total price, thereby completing the ship repair quotation.

Images