Coastal bulk cargo paper SOF receipt identification method and system based on large model

Abstract

The invention discloses a coastal bulk cargo paper SOF document identification method and system based on a large model, and relates to the technical field of image information identification, and the method comprises the steps: taking a paper SOF document image, employing an image detection algorithm of an OCR module to detect a text region of the paper SOF document image, obtaining the coordinate information of each text box, and carrying out the recognition of the coordinate information of each text box; performing direction correction and character recognition on the textbox, and outputting an original recognition result containing text content and corresponding space coordinates; performing joint coding on an original recognition result in the textbox coordinate fusion model, constructing a text association graph by utilizing a geometrical relationship and semantic coherence between textboxes, and generating a structured representation with spatial position features; extracting key fields of a main table based on the structured representation by adopting an SOF extraction model, complementing missing information of the table, extracting time points and discrete information, and outputting final structured data; according to the invention, the information identification efficiency and identification accuracy of the paper SOF document are improved.

Description

Technical Field

[0001] This application relates to the field of image information recognition technology, and in particular to a method and system for recognizing paper SOF documents for coastal bulk cargo based on a large model. Background Technology

[0002] In the coastal bulk cargo transportation sector, the SOF (Statement of Facts) document is the core document that records the facts of various operations of a vessel while in port. It is usually in paper form and contains a large amount of structured and semi-structured information, such as time points, operation content, quantity and unit.

[0003] Currently, although OCR technology can recognize information from paper SOF documents, it still faces significant challenges in processing SOF documents, resulting in poor information recognition efficiency and accuracy. On the one hand, SOF documents have diverse formats, with significant differences in formats used by different ports or shipping companies, and often contain handwritten content, stamps, and stains, which seriously affect the recognition effect of OCR. On the other hand, existing OCR technology struggles to effectively parse the complex table structures in documents, only able to extract text content without understanding the spatial relationships between text, leading to incomplete and inaccurate extraction of table information.

[0004] Therefore, how to improve the efficiency and accuracy of information recognition for paper SOF documents and reliably apply them to actual business scenarios such as coastal bulk cargo transportation has become an urgent technical problem to be solved. Summary of the Invention

[0005] To improve the efficiency and accuracy of information recognition for paper SOF documents and meet the actual business application needs of coastal bulk cargo transportation, this application provides a method and system for recognizing paper SOF documents for coastal bulk cargo based on a large model.

[0006] Firstly, the objective of this invention is achieved through the following technical solution: A method for identifying paper SOF documents for coastal bulk cargo based on a large model includes: The paper SOF document image is acquired, and the text region of the paper SOF document image is detected by the deep learning-based image detection algorithm in the OCR module. The coordinate information of each text box is obtained, and the text box is oriented and recognized. The original recognition result containing the text content and corresponding spatial coordinates is output. In the preset text box coordinate fusion model, the original recognition results are jointly encoded, and the text association graph is constructed by utilizing the geometric relationship and semantic coherence between text boxes to generate a structured representation with spatial location features. The SOF extraction model, constructed using a large language model, sequentially extracts key fields from the main table, completes missing information in the list, extracts time points and discrete information based on the structured representation, and outputs the final structured data.

[0007] By adopting the above technical solution, paper document information is efficiently and accurately converted into structured data to meet the stringent data processing requirements of scenarios such as port logistics. The recognition technology of this application consists of three core parts: an OCR module, a text box coordinate fusion model, and a SOF extraction model. These components work together to complete the conversion process from paper SOF document images to structured data. Specifically, the OCR module first uses a deep learning-based image detection algorithm to accurately locate the text regions in the paper SOF document image, obtaining the coordinate information of each text box and solving the problem of document format variability. Text correction eliminates the problem of inconsistent text directions, and finally outputs the original recognition result with spatial coordinates. Then, the text box coordinate fusion model performs joint encoding based on the text content and the corresponding spatial coordinates. It uses the geometric relationship and semantic coherence between text boxes to construct a text association graph with spatial topological relationships, upgrading the discrete text of traditional OCR into structured identification information. This effectively solves the shortcomings of the lack of table structure recognition in the existing technology, enabling the SOF extraction model to understand implicit logical relationships such as "quantity-unit-time", significantly improving the accuracy of table content recognition and extraction. Then, leveraging the powerful semantic understanding capabilities of the large language model, the fused structured representation is parsed to accurately and efficiently extract various key information from the SOF document, including key fields in the main table, missing information in the completion list, extracted time points, and discrete information. This achieves a comprehensive structured transformation of the SOF document, providing high-quality, standardized data output for subsequent data applications. Thus, this application achieves the technical effect of enhancing the information recognition efficiency and accuracy of paper SOF documents, and can be reliably applied to practical business scenarios such as coastal bulk cargo transportation and port logistics.

[0008] In a preferred embodiment of this application: the image detection algorithm includes the YOLOv8 algorithm or the DBNet algorithm; the deep learning-based image detection algorithm in the OCR module detects the text region of the paper SOF document image, obtains the coordinate information of each text box, performs orientation correction and character recognition on the text boxes, and outputs the original recognition result containing the text content and corresponding spatial coordinates, specifically including: The YOLOv8 algorithm or DBNet algorithm is used to detect text regions in the paper SOF document image to obtain multiple text box images, and the four-point coordinate information of each text box is output. The detected text box images are classified and corrected for 0°, 90°, 180° or 270° orientation using a text orientation classifier built on a lightweight convolutional neural network. A sequence recognition model is used to perform text recognition on the corrected text box image, output the text content, and form the original recognition result containing the text content and corresponding spatial coordinates.

[0009] By adopting the above technical solutions, the YOLOv8 algorithm or DBNet algorithm has excellent multi-scale feature extraction capabilities, and can simultaneously detect text boxes of different sizes such as headers and list items, with a text recall rate of 96.7%, making it suitable for various formats of paper SOF documents. By using a lightweight CNN-based orientation classifier to perform fine orientation correction on tilted text boxes, the OCR recognition accuracy is greatly improved, thereby significantly enhancing the recognition accuracy of complex SOF documents.

[0010] In a preferred embodiment of this application: the original recognition result is jointly encoded in a preset text box coordinate fusion model, and a text association graph is constructed using the geometric relationships and semantic coherence between text boxes to generate a structured representation with spatial location features, specifically including: Calculate the center point position based on the four-point coordinate information of each text box; Based on the location of the center point, the K nearest text boxes in space are selected as neighborhood nodes for each text box according to the K nearest neighbor algorithm to construct a text spatial relationship graph; Based on the actual annotation values ​​of the table structure, add a row-related label or a column-related label to each edge of the spatial relationship graph; The system integrates four types of features—text content semantic features, four-point coordinates of the text box and local image features, text format features, and information about the envelope line around the text box—to predict relationships and generate a structured representation with row and column topological relationships.

[0011] By adopting the above technical solution, a four-step strategy based on center point localization, K-nearest neighbor mapping, relation label injection, and multimodal feature fusion is used to transform discrete text into a structured graph. The spatial relationship graph constructed based on the center point and K-nearest neighbor algorithms can adapt to SOF documents with different resolutions and layout styles. Introducing realistically labeled row / column tags guides the model to learn the logical structure of tables. The fusion of four types of features—textual semantics, coordinates, format, and envelope—enhances the topological recognition capability for complex situations such as "spanning pages" and "merged cells."

[0012] In a preferred example, the SOF extraction model constructed using a large language model extracts key fields from the main table, completes missing information in the list, extracts time points and discrete information based on the structured representation, and outputs the final structured data, specifically including: The SOF extraction model includes an SOF main table extraction module, a list missing information completion module, an SOF time point extraction module, and an SOF other discrete information extraction module. The SOF main table extraction module extracts the ship name, voyage number, port, agent company, and cargo type fields through context prompts. The missing information completion module identifies missing information in the list based on contextual reasoning using a large language model, and completes the operation time, cargo type, and cargo quantity information based on the missing information in the list. The SOF time point extraction module extracts berthing time, departure time, and start and end times of operations, and converts them into ISO standard time format; The SOF other discrete information extraction module extracts notes, signatures, special instructions or abnormal situation records; Based on the SOF main table extraction module, the list missing information completion module, the SOF time point extraction module, and the SOF other discrete information extraction module, the final structured data is output.

[0013] By adopting the above technical solutions, layered and accurate extraction of structured information is achieved. The SOF main table extraction module utilizes the contextual understanding capabilities of the large language model, combined with prompting engineering, to accurately locate key fields such as ship names and ports. Even if field labels are ambiguous or missing, semantic association can be used for inference. The list missing information completion module utilizes the contextual reasoning capabilities of LLM to automatically complete missing items such as cargo quantities and operation times caused by occlusion or omission. The SOF time point extraction module converts the data to the ISO standard format to avoid time ambiguity. The remaining discrete information extraction modules of SOF capture unstructured information such as signatures and remarks, greatly improving the completeness of the final structured data output.

[0014] In a preferred embodiment, this application further includes: Obtain a sample library of historical paper SOF documents, identify key area image data in the historical paper SOF document images, the key areas include ship name area, voyage area, cargo description area and loading and unloading time area, each key area is associated with an area identifier and corresponds to at least one structured information field; Based on the region identifier, historical recognition datasets of each key region are obtained, and training and optimization based on field recognition accuracy are performed in the preset initial large model recognition engine to obtain an optimized large model recognition engine. Based on the historical paper SOF document image sample library, the semantic association factors between each key area are calculated and determined, and the semantic association feature vectors of all associated areas are generated. Based on the semantic association feature vector and the corresponding recognition parameter adjustment strategy of the historical recognition dataset, the maximum recognition confidence of each key region in the corresponding structured information field is determined, and the optimal recognition strategy is generated. The SOF extraction model performs structured information recognition and extraction on the paper SOF documents to be processed based on the optimal recognition strategy.

[0015] By adopting the above technical solution, this application introduces a historical data-driven adaptive optimization method to achieve continuous optimization of the recognition technology. Based on historical samples and historical recognition datasets, by analyzing the historical SOF document sample library, key areas are identified and a field-level recognition dataset is established to optimize the large model recognition engine and improve the recognition accuracy of specific fields (such as voyage number and agency company). The introduction of semantic association factors enables the system to enhance the confidence of field matching based on common combinations such as "ship name-voyage" and "cargo-port". By generating an optimal tenfold strategy, the recognition threshold and parameters of each area can be dynamically adjusted to maximize confidence and significantly improve the overall recognition stability.

[0016] In a preferred embodiment, after acquiring a sample library of historical paper SOF document images and identifying key area image data in the historical paper SOF document images, the method further includes: Based on the sample image information of various key regions and the corresponding recognition result data, multiple sample recognition difficulty intervals are created; based on the multiple sample recognition difficulty intervals, the sample image information of various key regions and the corresponding historical recognition result data are divided according to the region identifier, and multiple historical recognition performance data of various key regions in the recognition process are obtained. Based on the recognition accuracy data in multiple historical recognition performance data and the preset recognition difficulty judgment conditions, different recognition difficulty levels for various key areas are determined accordingly. The associated feature vector is updated based on the different recognition difficulty levels.

[0017] By adopting the above technical solutions, a sample recognition difficulty range is established, and historical performance data is classified and statistically analyzed in conjunction with regional identifiers. The system can quantitatively evaluate the recognition difficulty of different key areas (such as handwritten ship names and abbreviated cargo types). Based on the accuracy and preset conditions, the recognition difficulty level is determined, providing a basis for subsequent differentiated processing. For example, high-difficulty areas can trigger more complex post-processing logic or manual review prompts to achieve refined management of recognition difficulty.

[0018] In a preferred embodiment of this application: the key areas further include port areas and operational information areas; the semantic association factors include cargo information association factors, time information association factors, port information association factors, and operational information association factors; the step of calculating and determining the semantic association factors between each key area based on the historical paper SOF document image sample library, and generating semantic association feature vectors for all associated areas, specifically includes: Based on the historical paper SOF document image sample library, the historical recognition confidence thresholds and corresponding semantic association data for the following types of key areas were obtained: Semantic association data between the ship name area and the voyage area, semantic association data between the cargo description area and the port area, and semantic association data between the loading and unloading time area and the operation information area; Based on the contextual logical relationship between each key region and combined with the corresponding semantic association data, the weight coefficient value of the semantic association factor of each key region is calculated and determined. Based on the historical identification confidence threshold of each key region, the semantic association feature vector of all key regions is calculated by weighted summation.

[0019] By employing the aforementioned technical solution, and analyzing the co-occurrence patterns and contextual dependencies of historical data in areas such as "ship name-voyage," "cargo-port," and "time-operation," weighted semantic association data is constructed. This enables the system not only to recognize text but also to understand reasonable combinations between fields. A weighted summation based on historical recognition confidence thresholds generates a semantic association feature vector, which guides field matching and error correction in current documents. For example, when "coal" cargo appears, the system is more inclined to match typical coal ports such as "Qinhuangdao" and "Huanghua," or other user-predefined port fields, reducing the risk of mismatches.

[0020] Secondly, the objective of this invention is achieved through the following technical solution: A large-model-based system for recognizing paper SOF documents for coastal bulk cargo is applied to the large-model-based method for recognizing paper SOF documents for coastal bulk cargo described above. The system includes: The image acquisition and OCR processing module is used to acquire images of paper SOF documents, and to detect the text regions of the images using a deep learning-based image detection algorithm to obtain the coordinate information of each text box. The module performs orientation correction and character recognition on the text boxes and outputs the original recognition results containing the text content and corresponding spatial coordinates. The text box coordinate fusion module is used to jointly encode the original recognition results in a preset text box coordinate fusion model, and to construct a text association graph by utilizing the geometric relationship and semantic coherence between text boxes to generate a structured representation with spatial location features. The SOF information extraction module uses an SOF extraction model built with a large language model. Based on the structured representation, it sequentially extracts key fields from the main table, completes missing information in the list, extracts time points and discrete information, and outputs the final structured data.

[0021] Thirdly, the objective of this invention is achieved through the following technical solution: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for recognizing paper SOF documents for coastal bulk cargo based on a large model.

[0022] Fourthly, the objective of this invention is achieved through the following technical solution: A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the coastal bulk cargo paper SOF document recognition method based on a large model as described above.

[0023] In summary, this application includes at least one of the following beneficial technical effects: 1. This application adopts a three-level collaborative architecture based on deep learning-based OCR module preprocessing, coordinate fusion modeling, and large language model extraction to achieve high-precision structured recognition of complex layouts and handwritten interference in coastal SOF paper documents. The OCR module preprocessing stage can obtain the original recognition results with text content and corresponding spatial coordinates. Then, the text box coordinate fusion model uses geometric relationships and semantic coherence to construct a text association graph, effectively solving the field misalignment problem caused by table breaks and cross-line merging in traditional OCR. The introduction of a large language model for context-aware information extraction significantly improves the ability to reason and complete fuzzy and missing information. The entire process takes into account both spatial layout understanding and semantic logical reasoning, overcoming the structural restoration problem of existing OCR systems on non-standard documents. 2. High recall text region localization is achieved using YOLOv8 or DBNet, which is especially suitable for dense tables and italic text scenarios in SOF documents; a lightweight orientation classifier quickly identifies and corrects four-way rotations of 0°, 90°, 180°, and 270°, solving the problem of recognition failure caused by tilted writing during manual filling; 3. A four-step strategy—"center point localization, K-nearest neighbor graph construction, relation label injection, and multimodal feature fusion"—is employed to transform discrete text into a structured graph. A spatial relationship graph is constructed based on the center point and K-nearest neighbor algorithms, enabling it to adapt to SOF documents with different resolutions and layout styles. Real-world labeled row / column tags are introduced to guide the model in learning the logical structure of the table. Attached Figure Description

[0024] Figure 1 This is a flowchart of a method for identifying paper SOF documents for coastal bulk cargo based on a large model in one embodiment of this application; Figure 2 This is another flowchart in one embodiment of the coastal bulk cargo paper SOF document recognition method based on a large model in this application. Detailed Implementation

[0025] The present application will be further described in detail below with reference to the accompanying drawings.

[0026] In one embodiment, such as Figure 1 As shown, this application discloses a method for recognizing paper SOF documents for coastal bulk cargo based on a large model, which specifically includes the following steps: S1: Obtain the image of the paper SOF document, use the deep learning-based image detection algorithm in the OCR module to detect the text region of the paper SOF document image, obtain the coordinate information of each text box, perform orientation correction and character recognition on the text boxes, and output the original recognition result containing the text content and corresponding spatial coordinates.

[0027] In this embodiment, the deep learning-based image detection algorithm includes the YOLOv8 algorithm or the DBNet algorithm; the OCR module in this embodiment is used to locate text regions and recognize content through deep learning image detection algorithms, solving the recognition problem caused by creases and stains on paper documents.

[0028] Specifically, step S1 includes: S11: Use the YOLOv8 algorithm or DBNet algorithm to perform text region detection on the paper SOF document image, obtain multiple text box images, and output the four-point coordinate information of each text box.

[0029] In this embodiment, the four-point coordinate information refers to the coordinates of the four corner points of the text box, including the coordinates of the top left, top right, bottom right, and bottom left.

[0030] Specifically, an OCR detection model based on the YOLOv8 or DBNet algorithm was pre-trained and configured in the OCR module. During model training, 10,000 images of coastal bulk cargo SOF paper documents were collected, covering different port formats and standard text area four-point coordinates, such as "ship name area coordinates [(15, 25), (120, 25), (120, 60), (15, 60)]". YOLOv8 was used for training, with an input size of 640×640. Anchor box parameters were optimized to adapt to small text areas such as voyage number areas. The confidence threshold of YOLOv8 was 0.7, and the non-maximum suppression (NMS) IoU threshold was 0.5. For stamped text such as "customs inspection stamp", dilated convolution was used to enhance feature extraction capabilities. For stained areas, Gaussian filtering was used for preprocessing to reduce noise.

[0031] S12: The detected text box images are classified and corrected for 0°, 90°, 180° or 270° orientations using a text orientation classifier built on a lightweight convolutional neural network.

[0032] In this embodiment, a lightweight convolutional neural network (CNN) is used to identify the rotation angle (0°, 90°, 180°, 270°) of the text box image, thus solving the problem of text recognition errors caused by scanning tilt and handwritten skew.

[0033] Specifically, the text orientation classifier model architecture includes a 4-layer CNN classifier, with 3×3 convolutional kernels and [16, 32, 64, 128] channels, using the ReLU activation function. The fully connected layers output 4-dimensional vectors, corresponding to the four orientation categories. The lightweight convolutional neural network uses affine transformation matrices for correction during training.

[0034] S13: Use a sequence recognition model to perform text recognition on the corrected text box image, output the text content, and form the original recognition result containing the text content and corresponding spatial coordinates.

[0035] In this embodiment, a CRNN or SVTR sequence recognition model is used. A sequence-to-sequence model is used to recognize the text content in the corrected text box. The text content and spatial coordinates are fused to generate a structured original recognition result, which solves the recognition problem of complex scenarios such as handwriting and cursive writing.

[0036] For example, a sequence recognition model using a CRNN (CNN+BiLSTM+CTC) architecture is employed. The CNN uses a ResNet-18 backbone for feature extraction, the BiLSTM sequence modeling is based on a 128-unit bidirectional LSTM, and then the character sequence is decoded and output using CTC. During model training, domain-specific training is required, such as adding bulk cargo transportation-specific terms like "iron ore powder," "tonnage (T)," and "manifest number," and employing a number-first decoding strategy for cargo quantities (e.g., "5000T") to enhance number recognition capabilities. This embodiment's sequence recognition model, based on orientation-corrected text box images, can output recognition results and associate them with corresponding spatial coordinates.

[0037] Furthermore, the sequence recognition model is configured with error correction rules, such as requiring the flight number to contain the prefix "Voy", and automatically correcting the sequence recognition result "Vy123" to "Voy123".

[0038] S2: In the preset text box coordinate fusion model, the original recognition results are jointly encoded, and the text association graph is constructed by utilizing the geometric relationship and semantic coherence between text boxes to generate a structured representation with spatial location features.

[0039] In this embodiment, the text box coordinate fusion model is used to combine text content with spatial coordinates to generate a structured representation, enabling the SOF extraction model in step S3 to understand the implicit layout relationships of the table (such as the row relationship between "ship name" and "voyage"). Joint encoding is used to construct a text association graph based on geometric relationships and semantic coherence. Geometric relationships are determined by the K-nearest neighbor algorithm and the envelope line information around the text box, while semantic coherence is obtained through text features and row relationship labels and column relationship labels.

[0040] Specifically, step S2 includes: S21: Calculate the center point position based on the four-point coordinate information of each text box.

[0041] In this embodiment, the center point is located at the two-dimensional coordinate point (x). c y c ).

[0042] Specifically, the text box output by the OCR module contains the coordinates of four points (e.g., the coordinates of the ship name area: [(x1, y1), (x2, y2), (x3, y3), (x4, y4)]). The calculated center point coordinates are normalized to [0, 1] according to the image size.

[0043] S22: Based on the center point location, select the K nearest text boxes in space as neighborhood nodes for each text box according to the K nearest neighbor algorithm, and construct a text spatial relationship graph.

[0044] In this embodiment, the K-nearest neighbor algorithm selects the five spatially nearest text boxes to construct a relationship graph. K is set to 5. The K-nearest neighbor algorithm is used to quantify the geometric correlation of "nearby text boxes" and solve the problem of table structure discretization.

[0045] Specifically, calculate the Euclidean distance matrix of the center points of all text boxes, and select the 5 nearest neighborhood nodes for each text box, such as the voyage, port, and agency areas for the ship name area. Then, construct an undirected graph G = (V, E) with the text boxes as nodes and neighborhood relationships as edges. For example, the ship name node is connected to the voyage node (distance d = 100 pixels) and the port node (d = 120 pixels).

[0046] Furthermore, the K value can be adaptively adjusted according to the actual layout density of the SOF paper document, such as K value of 3 in sparse areas and K value of 8 in dense table areas.

[0047] S23: Based on the actual annotation values ​​of the table structure, add a row-related label or column-related label to each edge in the spatial relationship diagram.

[0048] In this embodiment, the Ground Truth value, combined with the same-line / same-column relationship labels, transforms geometric relationships into computable topological constraints. The same-line relationship label indicates whether the standard text box is located in the same row of the table. The presence of a same-line relationship label indicates that the text boxes are in the same row, and vice versa. Similarly, the same-column relationship label indicates whether the standard text boxes are located in the same column of the table. The presence of a same-column relationship label indicates that the text boxes are in the same column.

[0049] Specifically, the labeling rules include: if the y-coordinate deviation between the center points of two text boxes is ≤5 pixels (e.g., between a ship name area and a voyage area), they are considered to be in the same row; if the x-coordinate deviation between the center points of two text boxes is ≤5 pixels (e.g., between a cargo type area and a tonnage area), they are considered to be in the same column. The specific pixel thresholds can be customized according to the document layout. A traversal method is used to iterate through all edges of the spatial relationship diagram, adding labels based on coordinate deviations. If text boxes simultaneously meet the conditions of being in the same row / column, such as the intersecting cells of table headers, they are preferentially marked as "in the same row".

[0050] S24: Integrate four types of features—text content semantic features, four-point coordinates of the text box and local image features, text format features, and information about the envelope line around the text box—to predict relationships and generate a structured representation with row and column topological relationships.

[0051] In this embodiment, multimodal feature information, including text semantics, image, format, and geometric features, is integrated. A structured representation with row and column topological relationships is generated using a Graph Convolutional Network (GCN) to address the problem of fragmented cross-modal information. Different feature types employ different extraction methods. Text content semantic features utilize Word2Vec word vectors with a dimension of 128; local image features use the 512-dimensional feature vector output from the last convolutional layer of ResNet-18, such as the depth features of the ROI image for the ship name region; text format features include font size (12pt), bolding (1 / 0 standard, 1 for bold), and color depth (depth parameter 0-255), for example, [12, 1, 180]; the envelope line information around the text box uses Hough transform to detect the text box boundary line and calculates the line slope k and intercept b, with the upper boundary line having a slope k = 0.02 and b = 25. Furthermore, a graph convolutional network (GCN) is used to perform relational reasoning on the node features formed by concatenating four types of features. The relational reasoning formula is as follows: Where σ(·) is the activation function, introducing a nonlinear transformation; It is an adjacency matrix with self-loops (including row / column labels); W is the degree matrix, used to normalize the adjacency matrix and solve the problem of uneven distribution of node degree; (l) H is a trainable weight matrix; (l) H is the feature matrix of the nodes in the l-th layer. (l)The number of text boxes is used as the number of nodes; the GCN model achieves an F1 score of 96.3% on the port document test set, which is an improvement over the single-modal method. Where A is the original adjacency matrix, based on node connection relationships; I is the identity matrix used to add self-loops.

[0052] S3: The SOF extraction model is built using a large language model. Based on the structured representation, it sequentially extracts key fields from the main table, completes missing information in the list, extracts time points and discrete information, and outputs the final structured data.

[0053] In this embodiment, the SOF extraction model is based on parsing the structured representation using a large language model to achieve field extraction and missing information completion. The SOF extraction model includes an SOF main table extraction module, a list missing information completion module, an SOF time point extraction module, and a SOF other discrete information extraction module.

[0054] Specifically, step S3 includes: S31: Among them, the SOF main table extraction module extracts the ship name, voyage number, port, agent company and cargo type fields through context prompts.

[0055] In this embodiment, the contextual prompt is "Prompt Engineering." The Prompt design extracts parsed JSON data from the structured representation generated by the coordinate fusion model. The SOF main table extraction module sets a confidence threshold of 0.85; if it falls below this value, manual review is triggered. Spatial location features reduce the field binding error rate, such as preventing misalignment between "ship name" and "agency."

[0056] S32: The list missing information completion module identifies missing information in the list based on contextual reasoning using a large language model, and completes the operation time, cargo type, and cargo quantity information based on the missing information.

[0057] In this embodiment, based on the contextual reasoning capability of the Large Language Model (LLM), missing fields (tonnage, operation time, etc.) in the list area (such as the goods list) are automatically identified and completed, thus solving the problem of incomplete document information.

[0058] Specifically, first, a field missing detection is performed. If there is no number after the cargo type (e.g., "iron ore powder,"), it is marked as "tonnage missing". LLM semantic analysis: "Cargo: iron ore powder, [missing]; Operation time: [missing]", then the missing field = ["tonnage", "operation time"]. The average tonnage of similar cargoes is retrieved (e.g., "iron ore powder" is completed to "5000T"). If the previous operation item is "Bug 2: Coal, 3000T", then "Bug 3: Iron ore powder, 5000T" is completed. If there are multiple candidate solutions (e.g., tonnage "5000T" or "6000T"), the value with the highest confidence (>0.9) is selected.

[0059] S33: The SOF time point extraction module extracts berthing time, departure time, and start and end times of operations, and converts them into ISO standard time format.

[0060] In this embodiment, unstructured time descriptions are extracted by regular expression matching, and after time logic verification, they are uniformly converted into the ISO 8601 standard format. Time zones are also automatically added according to the port location.

[0061] S34: The SOF other discrete information extraction module extracts notes, signatures, special instructions or abnormal situation records.

[0062] In this embodiment, the abnormal situation record includes port operation interruption events such as equipment failure, weather delays, and cargo abnormalities.

[0063] Specifically, Named Entity Recognition (NER) technology is used to locate the boundaries of fields such as "remarks" and "signatures" and to classify the text. If the text contains "fault", "delay", or "abnormality", it is marked as an "abnormal record". For the signature area, the signature area image is extracted and its authenticity is verified using a CNN.

[0064] S35: Based on the SOF main table extraction module, the list missing information completion module, the SOF time point extraction module, and the SOF other discrete information extraction module, the final structured data is output.

[0065] Specifically, the outputs of the four modules are combined to generate standardized JSON or XML data that conforms to the port business system. When there is a conflict between the main table and the list, such as a discrepancy in port names, the field in the main table takes precedence and an alarm message is triggered to the user terminal.

[0066] Furthermore, the SOF extraction model uses a multi-dimensional test set during training and testing, such as SOF paper document images with different port formats, degrees of soiling, and handwriting quality. Model quantitative evaluation metrics include calculating the accuracy of main table field extraction, the recall rate for list information completion, and the F1 score for time-point recognition. Dynamic parameter tuning includes iterative optimization of the text detection threshold of the OCR module, the K-nearest neighbor parameters of the coordinate fusion model, and the Prompt template of the SOF extraction model based on the evaluation results.

[0067] In one embodiment, such as Figure 2 As shown, the method for identifying paper SOF documents for coastal bulk cargo based on a large model also includes: S10: Obtain a sample library of historical paper SOF document images, identify key area image data in the historical paper SOF document images, key areas include ship name area, voyage area, cargo description area and loading / unloading time area, each key area is associated with an area identifier and corresponds to at least one structured information field.

[0068] In this embodiment, the historical paper SOF document sample library refers to a collection of manually annotated SOF paper document images, including image data, four-point coordinate annotations for key areas, and structured field mappings. The structured field mappings refer to the binding relationship between each area and a field, such as the ship name area corresponding to "vessel_name". Key area image data refers to the image areas in the document that carry core business information. Key areas also include port areas and operational information areas. The area identifier is a unique code for the key area. The structured information fields are standard business fields extracted from the key areas.

[0069] Specifically, the voyage area is the area that marks the voyage number; the cargo description area is the area that marks the cargo type and specifications; the loading and unloading time area is the area that marks the start and end times of cargo loading and unloading; and the area identifier is used to associate structured fields.

[0070] Specifically, after image preprocessing of historical paper SOF document images, an improved DBNet algorithm (adjusting the binarization threshold α = 0.3 to adapt to the mixed printed and handwritten scenarios of SOF paper documents) is used to detect all text regions and output candidate box coordinates. Then, based on domain knowledge, key regions are selected through a combination of manual and rule-based matching for key region annotation. The domain knowledge references the "Standard Format of Coastal Bulk Cargo Transportation Documents" JT / T 1234-2023.

[0071] S20: Obtain historical recognition datasets for each key region based on the region identifier, and perform training and optimization based on field recognition accuracy in the preset initial large model recognition engine to obtain an optimized large model recognition engine.

[0072] In this embodiment, the historical recognition dataset consists of paired data of key region images and field text, including region images, structured field text, and corresponding recognition confidence scores. The initial large-scale model recognition engine is a pre-trained multimodal model LayoutLMv3, which is an enhancement component of the OCR module responsible for simultaneously processing image pixels and text sequences. That is, the output of LayoutLMv3 (text + coordinates) serves as the input to the coordinate fusion model. Training optimization refers to fine-tuning the model parameters based on the field recognition accuracy. The training objective is to minimize the field recognition error rate.

[0073] Specifically, high-quality samples with a confidence level greater than or equal to 0.8 are selected, and the loss function is designed based on different weights associated with different fields, such as a weight of 1.5 for the ship name field and a weight of 1.2 for the tonnage field.

[0074] S30: Based on the historical paper SOF document image sample library, calculate and determine the semantic association factors between key areas, and generate semantic association feature vectors for all associated areas.

[0075] In this embodiment, the semantic association factor is a parameter that quantifies the logical relationship between key regions, including the spatial association of the Euclidean distance between the regional center points and the semantic co-occurrence of the field co-occurrence frequency; the semantic association feature vector is a 128-dimensional vector that encodes the strength and type of the association between regions, where the vector dimension 0-63 represents the spatial relationship, the dimension 64-95 represents the semantic relationship, such as the field type matching degree, and the dimension 96-127 represents the layout rule compliance degree.

[0076] Specifically, step S30 includes: S301: Based on the historical paper SOF document image sample library, the historical identification confidence thresholds and corresponding semantic association data for the following types of key areas are obtained: S3011: Semantic association data between the ship name area and the voyage area, semantic association data between the cargo description area and the port area, and semantic association data between the loading / unloading time area and the operation information area.

[0077] In this embodiment, the semantic association factors include cargo information association factors, time information association factors, port information association factors, and operational information association factors. The cargo information association factor applies to the cargo description area and the port area, calculated based on the matching degree between cargo type and port facilities and vessel load. The time information association factor applies to the loading / unloading time area and the operational information area, calculated based on the correlation between operation time and tide tables and port operating hours. The port information association factor applies to the port area and the voyage area, calculated based on the topological relationship between port geographical location and shipping route. The operational information association factor applies to the operational information area and the cargo description area, calculated based on the adaptability of operational instructions and cargo characteristics.

[0078] Specifically, based on the fully annotated SOF paper document images in the historical sample database, the association pairs between ship name and voyage are extracted (by verifying the binding relationship between ship name and voyage through the ship AIS database), the association pairs between cargo and port are extracted (by statistically analyzing the high-frequency cargo types of each port based on the port cargo throughput database), and the association pairs between time and operation information are extracted. Time-sensitive operation instructions are extracted by parsing the operation logs.

[0079] S3012: Based on the contextual logical relationship between each key region and combined with the corresponding semantic association data, calculate and determine the weight coefficient value of the semantic association factor for each key region.

[0080] In this embodiment, the weighting coefficient value ω k The calculation formula is: Where k represents the key region type; the association confidence score is the field matching accuracy rate verified by manual verification, such as ship name-voyage association confidence score = 98%.

[0081] For example, the weighting coefficient ω between each pair of regions k The values ​​for the corresponding pairings of vessel name area and voyage area, cargo area and port area, and time area and operation information area are 0.35, 0.28, 0.22, and 0.15, respectively.

[0082] S3013: Based on the historical identification confidence threshold of each key region, the semantic association feature vector of all key regions is calculated by weighted summation.

[0083] For example, the historical confidence threshold for the ship name area is 0.92; the historical confidence threshold for the port area is 0.85; the historical confidence threshold for the operation information area is 0.80; and the historical confidence threshold for other areas is preset to 0.86. The historical confidence thresholds are statistically derived to determine the minimum confidence requirement for each area.

[0084] Specifically, the strong correlation between ship name and voyage (ρ = 0.98) is amplified with high weight, causing the model to prioritize the spatial proximity between the two (e.g., the voyage is usually 20-50 pixels to the right and below the ship name). When the confidence level of voyage recognition is <0.92, the ship name region context verification is automatically invoked, reducing the voyage recognition error rate from 12% to 7.2% (based on test set statistics).

[0085] The feature fusion formula is: Where ρ k Cosine similarity is calculated using CLIP as the semantic association factor for region pairs; Conf kThis is the historical confidence threshold for the region pair. The output semantic association feature vector is a 128-dimensional semantic association feature vector. For example, V = [0.35*0.98, 0.28*0.82, 0.22*0.75, ...] = [0.343, 0.230, 0.165, ...].

[0086] S40: Adjust the recognition strategy based on the semantic association feature vector and the recognition parameters corresponding to the historical recognition dataset, determine the maximum recognition confidence of each key region in the corresponding structured information field, and generate the optimal recognition strategy.

[0087] In this embodiment, the identification parameter adjustment strategy is a rule set for dynamically optimizing the identification process, including confidence threshold, processing priority, and completion priority. The confidence threshold can be set differently for different key areas, such as ≥0.85 for the ship name area and ≥0.75 for the cargo area. The processing priority is to identify the ship name first, then the voyage number; the completion trigger condition is to call the historical average when the tonnage segment is missing. The optimal identification strategy is the most efficient identification scheme generated by combining historical data and real-time analysis, including batch identification of highly correlated areas, downgrading of low-confidence areas, and automatic completion of missing fields.

[0088] Specifically, the maximum recognition confidence score is calculated based on the semantic feature weights (0.6), historical data weights (0.4), and parameter combination similarity (the inverse of the Euclidean distance) of each key region to obtain the theoretical maximum confidence score for each key region under optimal parameters. For example, the maximum recognition confidence score for the port region is 0.96. The optimal recognition parameters are compared with the current recognition confidence score. If the current recognition confidence score is greater than or equal to the maximum recognition confidence score, the current parameters are maintained; otherwise, parameter search is initiated, and neighborhood parameter sampling is performed to calculate a new recognition confidence score. If the recognition confidence score of the new recognition parameters is greater than the old recognition confidence score, the new recognition parameters are adopted. The neighborhood sampling rule is based on the semantic inference parameters and feature fusion weight coefficients, uniformly sampling 16 points within a radius R = 0.3.

[0089] S50: The SOF extraction model uses the optimal identification strategy to identify and extract structured information from paper SOF documents to be processed.

[0090] In this embodiment, the optimal identification strategy is the SOF extraction model as the intelligent control center of the identification process.

[0091] Specifically, the port business rule base in the optimal identification strategy provides domain knowledge support for the SOF main table extraction module. For example, when identifying the ship name region, if the confidence level is <0.95 (strategy threshold), the system automatically calls the ship database to verify the "ship name-voyage" binding relationship. When the main table extraction module extracts fields through Prompt Engineering, the semantic association factor (e.g., ship name and voyage ρ = 0.98) can significantly improve the accuracy of key field matching. Under the guidance of the optimal identification strategy, the list missing information completion module prioritizes using large model derivation to handle common missing information. If low confidence is encountered, it automatically switches to the rule base to improve the success rate of information completion.

[0092] In this embodiment, after acquiring a sample library of historical paper SOF documents and identifying key area image data in the historical paper SOF document images, the method further includes: S100: Based on the sample image information of various key areas and the corresponding recognition result data, multiple sample recognition difficulty ranges are created.

[0093] In this embodiment, the sample recognition difficulty range is a subset of samples divided according to image quality (clarity / staining degree) and text complexity (font or layout).

[0094] Specifically, the image processing quality of key regions is evaluated and calculated based on formula (1) to optimize the generation of recognition strategies: Among them, Q i C is the image quality assessment value for the key region identified as i, with a value ranging from 0 to 1; i R is the sharpness coefficient of the region image; i D is the regularity coefficient of the region image; i N represents the distortion degree of the region image; i α represents the noise interference level of the regional image; α is the ambient light compensation coefficient, with a value ranging from 0.8 to 1.2. Based on the calculation results of formula (1), the image quality is divided into 1 to 4 levels, and the specific level classification criteria can be set as needed.

[0095] Furthermore, the text complexity of the document image is evaluated by analyzing the layout complexity coefficient λ, where λ = (number of non-parallel text boxes / total number of text boxes) × (proportion of handwritten text / 100); when λ is less than 0.3, it is a simple layout and is set as level A; when λ is greater than or equal to 0.3 and less than 0.6, it is a medium layout and is set as level B; when λ is greater than or equal to 0.6, it is a complex layout and is set as level C.

[0096] The difficulty range of sample recognition is generated by combining the image quality level (1-4) and the layout complexity level (AC) to generate 4×3=12 difficulty ranges, such as "Level 1-A" representing high-quality image + simple layout.

[0097] S200: Based on multiple sample recognition difficulty ranges, the sample image information of various key regions and the corresponding historical recognition result data are divided according to the region identifier, and multiple historical recognition performance data of various key regions in the recognition process are obtained.

[0098] In this embodiment, historical recognition performance data records the recognition indicators of the OCR module for each key area under different difficulty ranges. The recognition indicators include accuracy, recall, or F1 score.

[0099] Specifically, for samples within each difficulty range, three recognition metrics for each key region identified by the OCR module were statistically analyzed: Accuracy = Number of correctly recognized characters / Total number of characters; Recall = Number of correctly detected text boxes / Actual number of text boxes; F1 score = (2 × Accuracy × Recall) / (Accuracy + Recall). Performance databases were established separately for different key region types.

[0100] S300: Based on the recognition accuracy data from multiple historical recognition performance data and the preset recognition difficulty judgment conditions, different recognition difficulty levels for various key areas are determined.

[0101] In this embodiment, the difficulty level information refers to the difficulty labels (such as level 1-5) dynamically divided based on performance data, which are used to guide model resource allocation and parameter tuning.

[0102] Specifically, the criteria for determining the difficulty level are as follows: Condition 1: If the accuracy is less than 0.8 and the recall is less than 0.75, it is judged as difficulty level 5, extremely high difficulty; Condition 2: When 0.8 ≤ precision < 0.85 and 0.75 ≤ recall < 0.8, the difficulty level is determined to be 5, which is high difficulty; Condition 3: When 0.85 ≤ precision < 0.88 and 0.8 ≤ recall < 0.84, it is judged as difficulty level 3, medium difficulty; Condition 4: When 0.88 ≤ Precision < 0.92 and 0.84 ≤ Recall < 0.88, it is judged as difficulty level 2, low difficulty; Condition 5: If 0.92 ≤ Precision and 0.88 ≤ Recall, the difficulty level is 1, which is relatively low. If only one of the above conditions is met, the difficulty level is higher.

[0103] High-difficulty areas (levels 4-5) require more computing resources, while low-to-medium difficulty areas (levels 1-3) can use a lightweight model.

[0104] S400: Update the associated feature vector based on different recognition difficulty levels.

[0105] In this embodiment, the recognition difficulty level L∈[1, 5], where 1 is the simplest and 5 is the most complex; the original semantic association feature vector V_orig∈R^128 (128-dimensional vector) is updated with weight coefficients δ' based on different recognition difficulty levels: if L is 5, then δ'=1.2; if L is 4, then δ'=1.0; if L is 3, then δ'=0.8; if L is 2, then δ'=0.6; if L is 1, then δ'=0.4. High-difficulty regions (L≥4) have enhanced semantic association weights (δ>1) to compensate for image quality defects such as text blurring caused by dirt; while low-difficulty regions (L≤2) have reduced weights (δ<1), effectively avoiding over-reliance on semantic association, as the image is clear and can be directly recognized at this time; when the layout complexity in the image is greater than 0.6 or the image quality evaluation value is low, the update process of the association feature vector is triggered.

[0106] Specifically, the update formula for the associated feature vector is: V_new=V_orig×δ', where V_new is the updated associated feature vector.

[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0108] In one embodiment, a large-model-based coastal bulk cargo paper SOF document recognition system is provided, which corresponds to the large-model-based coastal bulk cargo paper SOF document recognition method in the above embodiments.

[0109] A large-scale model-based system for recognizing paper SOF documents for coastal bulk cargo includes modules a, b, c, and d. Detailed descriptions of each functional module are as follows: The image acquisition and OCR processing module is used to acquire images of paper SOF documents, and to detect the text region of the image using a deep learning-based image detection algorithm to obtain the coordinate information of each text box. The module performs orientation correction and character recognition on the text boxes and outputs the original recognition results containing the text content and corresponding spatial coordinates. The text box coordinate fusion module is used to jointly encode the original recognition results in a preset text box coordinate fusion model, and to construct a text association graph by utilizing the geometric relationship and semantic coherence between text boxes to generate a structured representation with spatial location features. The SOF information extraction module uses an SOF extraction model built with a large language model. Based on the structured representation, it sequentially extracts key fields from the main table, completes missing information in the list, extracts time points and discrete information, and outputs the final structured data. For specific limitations regarding the coastal bulk cargo paper SOF document recognition system based on a large model, please refer to the limitations of the coastal bulk cargo paper SOF document recognition method based on a large model mentioned above, which will not be repeated here. Each module in the above-mentioned coastal bulk cargo paper SOF document recognition system based on a large model can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in the processor of the computer device in hardware form or independent of the processor, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0110] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method for recognizing coastal bulk cargo paper SOF documents based on a large model.

[0111] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0112] In one embodiment, particularly according to embodiments of the invention, the processes described above with reference to the flowchart can be implemented as a computer software program. For example, embodiments of the invention include a computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the large-model-based coastal bulk cargo paper SOF document identification method. In such embodiments, the computer program can be downloaded and installed from a network via a communication module, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the various functions defined in this invention.

[0113] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0114] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for identifying paper SOF documents for coastal bulk cargo based on a large model, characterized in that, include: The paper SOF document image is acquired, and the text region of the paper SOF document image is detected by the deep learning-based image detection algorithm in the OCR module. The coordinate information of each text box is obtained, and the text box is oriented and recognized. The original recognition result containing the text content and corresponding spatial coordinates is output. In the preset text box coordinate fusion model, the original recognition results are jointly encoded, and the text association graph is constructed by utilizing the geometric relationship and semantic coherence between text boxes to generate a structured representation with spatial location features. The SOF extraction model, constructed using a large language model, sequentially extracts key fields from the main table, completes missing information in the list, extracts time points and discrete information based on the structured representation, and outputs the final structured data.

2. The method for identifying coastal bulk cargo paper SOF documents based on a large model according to claim 1, characterized in that, The image detection algorithm includes the YOLOv8 algorithm or the DBNet algorithm; the deep learning-based image detection algorithm in the OCR module detects the text region of the paper SOF document image, obtains the coordinate information of each text box, performs orientation correction and character recognition on the text boxes, and outputs the original recognition result containing the text content and corresponding spatial coordinates, specifically including: The YOLOv8 algorithm or DBNet algorithm is used to detect text regions in the paper SOF document image, resulting in multiple text box images, and the four-point coordinate information of each text box is output. The detected text box images are classified and corrected for 0°, 90°, 180° or 270° orientation using a text orientation classifier built on a lightweight convolutional neural network. A sequence recognition model is used to perform text recognition on the corrected text box image, output the text content, and form the original recognition result containing the text content and corresponding spatial coordinates.

3. The method for identifying coastal bulk cargo paper SOF documents based on a large model according to claim 2, characterized in that, In the preset text box coordinate fusion model, the original recognition results are jointly encoded, and a text association graph is constructed using the geometric relationships and semantic coherence between text boxes to generate a structured representation with spatial location features. Specifically, this includes: Calculate the center point position based on the four-point coordinate information of each text box; Based on the location of the center point, the K nearest text boxes in space are selected as neighborhood nodes for each text box according to the K nearest neighbor algorithm to construct a text spatial relationship graph; Based on the actual annotation values ​​of the table structure, add a row-related label or a column-related label to each edge of the spatial relationship graph; The system integrates four types of features—text content semantic features, four-point coordinates of the text box and local image features, text format features, and information about the envelope line around the text box—to predict relationships and generate a structured representation with row and column topological relationships.

4. The method for identifying coastal bulk cargo paper SOF documents based on a large model according to claim 1, characterized in that, The SOF extraction model, constructed using a large language model, sequentially extracts key fields from the main table, completes missing information in the list, extracts time points and discrete information based on the structured representation, and outputs the final structured data, specifically including: The SOF extraction model includes an SOF main table extraction module, a list missing information completion module, an SOF time point extraction module, and an SOF other discrete information extraction module. The SOF main table extraction module extracts the ship name, voyage number, port, agent company, and cargo type fields through context prompts. The missing information completion module identifies missing information in the list based on contextual reasoning using a large language model, and completes the operation time, cargo type, and cargo quantity information based on the missing information in the list. The SOF time point extraction module extracts berthing time, departure time, and start and end times of operations, and converts them into ISO standard time format; The SOF other discrete information extraction module extracts notes, signatures, special instructions or abnormal situation records; Based on the SOF main table extraction module, the list missing information completion module, the SOF time point extraction module, and the SOF other discrete information extraction module, the final structured data is output.

5. The method for identifying coastal bulk cargo paper SOF documents based on a large model according to claim 1, characterized in that, The method also includes: Obtain a sample library of historical paper SOF documents, identify key area image data in the historical paper SOF document images, the key areas include ship name area, voyage area, cargo description area and loading and unloading time area, each key area is associated with an area identifier and corresponds to at least one structured information field; Based on the region identifier, historical recognition datasets of each key region are obtained, and training and optimization based on field recognition accuracy are performed in the preset initial large model recognition engine to obtain an optimized large model recognition engine. Based on the historical paper SOF document image sample library, the semantic association factors between each key area are calculated and determined, and the semantic association feature vectors of all associated areas are generated. Based on the semantic association feature vector and the corresponding recognition parameter adjustment strategy of the historical recognition dataset, the maximum recognition confidence of each key region in the corresponding structured information field is determined, and the optimal recognition strategy is generated. The SOF extraction model performs structured information recognition and extraction on the paper SOF documents to be processed based on the optimal recognition strategy.

6. The method for identifying coastal bulk cargo paper SOF documents based on a large model according to claim 5, characterized in that, After acquiring a sample library of historical paper SOF documents and identifying key area image data in the historical paper SOF document images, the method further includes: Based on the sample image information of various key areas and the corresponding recognition result data, multiple sample recognition difficulty ranges are created. Based on the multiple sample recognition difficulty ranges, the sample image information and corresponding historical recognition result data of various key regions are divided according to the region identifier, thereby obtaining multiple historical recognition performance data of various key regions in the recognition process. Based on the recognition accuracy data in multiple historical recognition performance data and the preset recognition difficulty judgment conditions, different recognition difficulty levels for various key areas are determined accordingly. The associated feature vector is updated based on the different recognition difficulty levels.

7. The method for identifying coastal bulk cargo paper SOF documents based on a large model according to claim 5, characterized in that, The key areas also include port areas and operational information areas; the semantic association factors include cargo information association factors, time information association factors, port information association factors, and operational information association factors; the step of calculating and determining the semantic association factors between each key area based on the historical paper SOF document image sample library, and generating semantic association feature vectors for all associated areas, specifically includes: Based on the historical paper SOF document image sample library, the historical recognition confidence thresholds and corresponding semantic association data for the following types of key areas were obtained: Semantic association data between the ship name area and the voyage area, semantic association data between the cargo description area and the port area, and semantic association data between the loading and unloading time area and the operation information area; Based on the contextual logical relationship between each key region and combined with the corresponding semantic association data, the weight coefficient value of the semantic association factor of each key region is calculated and determined. Based on the historical identification confidence threshold of each key region, the semantic association feature vector of all key regions is calculated by weighted summation.

8. A coastal bulk cargo paper SOF document recognition system based on a large model, characterized in that, The system, applied to the large-model-based method for identifying paper SOF documents for coastal bulk cargo as described in any one of claims 1 to 7, comprises: The image acquisition and OCR processing module is used to acquire images of paper SOF documents, and to detect the text regions of the images using a deep learning-based image detection algorithm to obtain the coordinate information of each text box. The module performs orientation correction and character recognition on the text boxes and outputs the original recognition results containing the text content and corresponding spatial coordinates. The text box coordinate fusion module is used to jointly encode the original recognition results in a preset text box coordinate fusion model, and to construct a text association graph by utilizing the geometric relationship and semantic coherence between text boxes to generate a structured representation with spatial location features. The SOF information extraction module uses an SOF extraction model built with a large language model. Based on the structured representation, it sequentially extracts key fields from the main table, completes missing information in the list, extracts time points and discrete information, and outputs the final structured data.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the coastal bulk cargo paper SOF document identification method based on a large model as described in any one of claims 1 to 7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the coastal bulk cargo paper SOF document identification method based on a large model as described in any one of claims 1 to 7.

Images