Marine shipping rate text recognition method based on large language model

Abstract

In the method for identifying sea freight rate text based on a large language model provided in the application, a sea freight rate text is obtained, and a large language model is used to extract initial key information and remark information from the sea freight rate text. After the initial key information is corrected, the large language model is used to convert non-standard port names in the corrected key information into standard port names by using a port five-character code table to obtain converted key information. According to an obtained route corresponding table, the port corresponding to each voyage name in the converted key information is adjusted to obtain effective key information. The effective key information is supplemented with the remark information to obtain target key information. According to the target key information, a large language model is used to generate a plurality of entity corresponding relationships. After the large language model verifies the container type prices of ports with the same name in each entity relationship, a sea freight rate table is generated according to each entity relationship. This method improves the efficiency of extracting freight rate information from unstructured sea freight rate text.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a sea freight rate text recognition method based on a large language model. BACKGROUND

[0002] Under the background of rapid development of global trade, the international logistics industry bears increasingly heavy freight transportation demand, and the management and recognition of freight rate information play an important role. Especially in the field of sea freight, the complexity of freight rate information is significantly increased due to the changes of different regions, routes, and cargo types. However, since the freight rate usually exists in the form of unstructured text and involves a large amount of natural language description, its analysis and processing become a very challenging task.

[0003] The prior art relies on middle and senior personnel with rich industry experience to manually understand and convert freight rate text, and to analyze complex natural language content into structured information. This method not only consumes a lot of manpower, but also the conversion efficiency and accuracy are easily affected by the professional level of the operator, and it is difficult to adapt to large-scale and high-frequency data processing requirements. Under the current trend of digital transformation, the unstructured text form of sea freight rate is difficult to be programmed into structured data that can be directly used by the system, resulting in the need to rely on manual analysis, which makes the efficiency of sea freight rate text recognition low. SUMMARY

[0004] The purpose of the present application is to at least solve one of the above technical defects, in particular the technical defect of low efficiency of sea freight rate text recognition in the prior art.

[0005] In a first aspect, the present application provides a sea freight rate text recognition method based on a large language model, which comprises:

[0006] Obtaining sea freight rate text of international logistics, and using a large language model to extract initial key information and remark information from the sea freight rate text, wherein the initial key information includes the port of departure, the destination port, the ship name voyage, and the container type price;

[0007] Correcting the initial key information to obtain corrected key information, and using a large language model to convert non-standard port names in the corrected key information into standard port names by using a pre-obtained port five-character code table to obtain converted key information;

[0008] Obtaining a route correspondence table for representing the correspondence between ship name voyages and ports, adjusting the ports corresponding to each ship name voyage in the converted key information according to the route correspondence table to obtain effective key information, and supplementing the effective key information with the remark information to obtain target key information;

[0009] According to the target key information, a large language model is used to generate a plurality of entity corresponding relationships, and the large language model is used to verify the container type price of the same name port in each entity relationship, and a sea freight rate table is generated according to each entity relationship.

[0010] In one of the embodiments, the step of extracting initial key information and note information from the sea freight text by using the large language model includes:

[0011] The first prompt instruction is obtained, and the first prompt instruction is input into the large language model to obtain sea freight basic information, the sea freight basic information including the port of departure, the destination port and the ship name voyage;

[0012] According to the sea freight basic information, a second prompt instruction is determined, and the second prompt instruction is input into the large language model to obtain container type price information, and the sea freight basic information and the container type price information are taken as initial key information, and the container type price information includes the container type price of each port;

[0013] According to the sea freight basic information, a third prompt instruction is determined, and the third prompt instruction is input into the large language model to obtain note information, and the note information includes general notes of each port and special notes of specific ports.

[0014] In one of the embodiments, the step of correcting the initial key information to obtain the corrected key information includes:

[0015] Obtain the sea freight information correction data;

[0016] The initial key information is fuzzy matched with the sea freight information correction data to obtain a first matching result, and according to the first matching result, the spelling error information in the initial key information is modified, and the completely wrong information in the initial key information is removed to obtain the corrected key information.

[0017] In one of the embodiments, by using the large language model, the non-standard port name in the corrected key information is converted into a standard port name by using the pre-obtained port five-character code table to obtain the converted key information, and the step includes:

[0018] The port name in the corrected key information is matched with the port name in the port five-character code table to obtain a second matching result, and according to the second matching result, the non-standard port name in the corrected key information is replaced by the standard port name in the port five-character code table to obtain the converted key information.

[0019] In one of the embodiments, according to the route corresponding table, the port corresponding to each ship name voyage in the converted key information is adjusted to obtain the effective key information, and the step includes:

[0020] For each voyage in the converted key information, the voyage is matched with the route correspondence table to determine the respective transit ports corresponding to the voyage, and the initial port corresponding to the voyage that is inconsistent with the respective transit ports is removed to obtain valid key information.

[0021] In one of the embodiments, the step of verifying the box type price of the same-named port in each entity relationship by using the large language model comprises:

[0022] The consistency of the box type price of the same-named port in each entity relationship is verified by using the large language model, and if there is a difference in the box type price of the same-named port, the box type price with the difference is screened and corrected according to the preset rule.

[0023] In one of the embodiments, the step of generating the sea freight rate table according to each entity relationship comprises:

[0024] Each sub-table corresponding to each entity relationship is generated, and each sub-table is mapped by Cartesian product to obtain the sea freight rate table.

[0025] In a second aspect, the present application provides a sea freight rate text recognition device based on a large language model, which comprises:

[0026] The sea freight rate text acquisition module is used to acquire the sea freight rate text of international logistics, and the initial key information and the remark information are extracted from the sea freight rate text by using the large language model, the initial key information including the departure port, the destination port, the ship name voyage and the box type price;

[0027] The port name standardization module is used to correct the initial key information to obtain the corrected key information, and convert the non-standard port name in the corrected key information to the standard port name by using the large language model and the pre-acquired port five-code table to obtain the converted key information;

[0028] The target key information determination module is used to acquire the route correspondence table for indicating the correspondence between the ship name voyage and the port, adjust the port corresponding to each voyage in the converted key information according to the route correspondence table to obtain the valid key information, and supplement the valid key information by using the remark information to obtain the target key information;

[0029] The sea freight rate table generation module is used to generate a plurality of entity relationships by using the large language model according to the target key information, verify the box type price of the same-named port in each entity relationship by using the large language model, and generate the sea freight rate table according to each entity relationship.

[0030] In a third aspect, the present application provides a storage medium, the storage medium storing computer readable instructions, the computer readable instructions being executed by one or more processors to cause the one or more processors to perform the steps of the method for identifying sea freight rate text based on a large language model according to any one of the above embodiments.

[0031] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0032] The memory stores computer readable instructions, the computer readable instructions being executed by one or more processors to perform the steps of the method for identifying sea freight rate text based on a large language model according to any one of the above embodiments.

[0033] From the above technical solutions, the embodiments of the present application have the following advantages:

[0034] In the method for identifying sea freight rate text based on a large language model provided by the present application, the unstructured text content is preliminarily converted into structured data by obtaining the sea freight rate text and automatically extracting the initial key information and the remark information using the large language model; the non-standard port name in the initial key information is corrected and standardized converted using the large language model and the pre-obtained port five-character code table, so as to unify the port names into standardized expressions, which not only reduces the risk of human misjudgment, but also improves the uniformity and consistency of the information; after obtaining the correspondence table of ship name voyage and port, the correspondence between each voyage name and port in the converted key information is adjusted using the table, and the key information meeting the business logic is screened out; in addition, the effective key information is supplemented by the remark information, so that the information is more complete; according to the effective key information, the correspondence between multiple entities is generated using the large language model, which effectively reduces the errors caused by the intersection of complex relationships; and the data accuracy is ensured by verifying the box type price of the same name port; finally, the sea freight rate table is generated using the verified entity relationship, so that the originally unstructured rate information is converted into structured table data. Based on this, the method uses the natural language parsing ability of the large language model, combines the port five-character code table, the route correspondence table and other auxiliary information, and efficiently converts the originally manually recognized sea freight rate text into structured data, reduces the risk of human participation, significantly improves the efficiency and accuracy of data processing, and meets the high-frequency and high-precision processing requirements of the international logistics industry for rate information. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the accompanying drawings in the following description only represent some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0036] Figure 1 The flowchart of the method for identifying sea freight rate text based on a large language model provided by the embodiments of the present application is shown in the figure.

[0037] Figure 2 The structural diagram of the device for identifying sea freight rate text based on a large language model provided by the embodiments of the present application is shown in the figure.

[0038] Figure 3 The internal structure diagram of the computer device provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments only represent some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0040] The present application provides a method for identifying sea freight rate text based on a large language model. The following embodiments take the application of the method to a video platform as an example for illustration. It can be understood that the video platform can be various devices with data processing functions, which can be but not limited to a single server and a server cluster, etc. As shown in the figure, Figure 1 The method can include the following steps:

[0041] S101: Obtain the sea freight rate text of international logistics, and use a large language model to extract initial key information and note information from the sea freight rate text. The initial key information includes the port of departure, the destination port, the ship name voyage and the box type price.

[0042] The international logistics sea freight price text refers to unstructured text data containing international sea freight service price information, and usually includes information such as the port of departure and the port of destination, the ship name and voyage, the type of goods, the container type, and the freight price. These texts can come from multiple sources such as shipping companies and freight agents, and are usually in the form of natural language texts. The large language model (LLM) refers to a deep learning model with strong natural language processing capabilities, which can extract, analyze and structure information from unstructured text. The initial key information is the basic business elements extracted from the text, including the port of departure, the port of destination, the ship name and voyage, and the container type and price. The remark information is additional information related to the key information, usually including the conditions, restrictions or supplementary explanations of the freight price, which helps to improve and refine the description and analysis of the freight price.

[0043] In this step, the user input or uploaded sea freight price text can be received, and the obtained freight price text can be input into the LLM for information extraction. Illustratively, the LLM first performs word segmentation, part-of-speech tagging and syntactic analysis on the text, then locates and extracts the fields of the port of departure, the port of destination, the ship name and voyage, and the container type and price, and outputs the content of each field in a structured form. The large language model can use the COT (Chain of Thought) reasoning method, that is, it processes the text step by step to extract the key information and additional remark information, and the text information can be effectively processed in layers during this reasoning process to ensure the accurate extraction of key information and remark information.

[0044] It can be understood that the application of the large language model avoids the dependence on grammar in the traditional rule-based text parsing method, making the model more flexible and accurate in processing diverse and complex natural language descriptions. The LLM can accurately identify and extract key information even if the text format and content vary.

[0045] The remark information often contains special instructions for the freight price, which is crucial for business analysis. The COT method of the LLM can extract key information and supplementary remarks layer by layer, ensuring that the key information is complete and the remarks are clear, making the data more accurate in subsequent verification, analysis and generation processes. At the same time, this automated method reduces the error risk caused by human intervention and improves processing efficiency.

[0046] S102: correcting the initial key information to obtain corrected key information, and converting non-standard port names in the corrected key information into standard port names by using a pre-obtained port five-character code table through the large language model to obtain converted key information.

[0047] The corrected key information refers to the key information obtained after inspection, proofreading, and possible manual or automated correction. The port five-code table is a standardized mapping table containing port names, which maps the names of ports around the world to a unified five-code, including Chinese port names, English port names, and standard five-codes. This table is used to identify and unify different port names to conform to industry standards. The converted key information refers to the final key information obtained by replacing non-standard port names in the corrected key information with standardized port names using the port five-code table.

[0048] In this step, the initial key information can be automatically scanned to correct possible spelling or formatting errors by writing a checking function or enabling a large language model. The LLM can check the integrity and format consistency of the key information and make automatic completion and correction for obvious spelling errors or missing information. Then, by calling the port five-code table, non-standard port names in the corrected key information are matched and replaced with five-codes or standard port names.

[0049] It can be understood that, due to the possible spelling errors, formatting inconsistencies, and other problems in the freight text, correcting the initial key information through an automated manner can effectively improve the accuracy. Compared with manual correction, computer equipment can quickly process a large amount of text and reduce delays and errors caused by manual operation. Port name standardization is very important for accurately matching port information and performing freight statistics and data analysis. Different sources of freight texts may use different port expressions, even containing abbreviations or spelling errors. Using standard port names makes the information highly consistent and eliminates confusion and errors caused by inconsistent names.

[0050] S103: Obtain a route correspondence table for representing the correspondence between ship names and ports, and adjust the ports corresponding to each voyage name in the converted key information according to the route correspondence table to obtain valid key information. The valid key information is supplemented with remark information to obtain target key information.

[0051] The route correspondence table is a table recording the correspondence between ship voyages and the ports they pass through, including voyage, port, and arrival / departure time information. This table is used to identify the ports that a ship visits in a specific voyage, ensuring the accuracy of the voyage data. The valid key information is the key information that matches the actual voyage and port after being filtered according to the route correspondence table, excluding invalid port information to ensure that the data conforms to the actual situation of the voyage. The target key information is the final key information supplemented with remark information, which conforms to the actual situation of the voyage and contains necessary additional explanations, forming complete and accurate freight information.

[0052] In this step, the route correspondence table can be automatically obtained and updated through API calls, scheduled tasks, or database connections. For example, the latest route data can be extracted from a specified source at regular intervals and stored in a local database for subsequent queries. The converted key information is filtered and adjusted according to the route correspondence table. Then, the remark information is automatically associated with the valid key information. For example, by field matching or ID association, the remark information is attached to the corresponding valid key information item to form more complete target key information, ensuring that the remark information is synchronized when generating the freight table.

[0053] It can be understood that, since routes may change or update over time, obtaining the route correspondence table can ensure the accuracy and timeliness of the information. Adjusting the port correspondence relationship can avoid interference from invalid data. By filtering out valid ports that match the actual route, the accuracy of the key information can be ensured. In this way, both the data quality and the time cost of manual screening are improved, ensuring that the freight information and the actual route are consistent, which helps to improve the accuracy of the freight table. Remark information usually provides special applicable conditions or limitations, and supplementing these information can make the target key information more comprehensive. Associating remark information can improve accuracy and avoid missing key applicable conditions, thereby reducing misunderstandings and incorrect use.

[0054] S104: According to the target key information, a large language model is used to generate a plurality of entity correspondence relationships, and a large language model is used to verify the container type prices of ports with the same name in each entity relationship, and a sea freight table is generated according to each entity relationship.

[0055] Among them, the entity correspondence relationship is the relationship between different entities in data processing. For example, the relationship between the port and its corresponding container type price, the relationship between the voyage and the port, etc. Container type price refers to the freight price of different types of containers between a specific voyage and a port. The sea freight table is a structured data table generated according to the target key information and the entity correspondence relationship, which contains voyage, port of departure, destination port, container type, and corresponding freight information.

[0056] In this step, the target key information is analyzed by a large language model to automatically identify and extract the relationship between different entities, such as the shipping route between the port of departure and the destination port, the container type price of the port with the same name, etc. The system will use natural language processing (NLP) technology to convert the key information into a format that can be understood by machines to form a relationship graph or association matrix for subsequent analysis. Then, using a large language model, the container type price of the port with the same name is cross-verified according to the generated entity relationship. For example, by searching the database for price data that matches the target key information, it is ensured that the container type price of each port is consistent and accurate.

[0057] Once the verification is completed, the information can be integrated into a structured table by the data processing module according to the entity correspondence relationship and the verification result. For example, using a data visualization tool or a table processing library, a sea freight rate table containing information such as voyage, port of departure, destination port, container type, and freight rate is generated, and finally output in the required file format, such as CSV, Excel, etc.

[0058] It can be understood that the entity correspondence relationship generated by the large language model can effectively sort and organize the data structure and understand the internal relationship between different data. Verifying the container type price of the same named port ensures the reliability of the freight rate data. Since the container type price may be affected by market fluctuations, voyage differences, and other factors, the automatic verification mechanism can quickly identify and solve potential problems. The final generated sea freight rate table makes the data more clear and systematic, facilitating subsequent queries, statistics, and analysis. In addition, the automation of the table output process greatly improves work efficiency and reduces the cost and risk of manual operation.

[0059] In the above embodiment, by obtaining sea freight rate text and using a large language model to automatically extract initial key information and note information, unstructured text content is preliminarily converted into structured data; using a large language model and a pre-obtained port five-character code table, non-standard port names in the initial key information are corrected and standardized converted, and each port name is unified to a standardized expression, which not only reduces the risk of human misjudgment, but also improves the uniformity and consistency of information; after obtaining the correspondence table of ship name voyage and port, the table is used to adjust the port corresponding to each voyage name in the converted key information, and the key information that meets the business logic is selected, in addition, the note information is used to supplement the effective key information, making the information more complete; according to the effective key information, a large language model is used to generate the correspondence relationship between multiple entities, effectively reducing errors caused by complex relationship intersections, and verifying the container type price of the same named port to ensure data accuracy; finally, a sea freight rate table is generated using the verified entity relationship, which converts the originally unstructured freight rate information into structured table data. Based on this, the method uses the natural language parsing ability of the large language model, combined with the port five-character code table, the route correspondence table and other auxiliary information, to efficiently convert the originally manually recognized sea freight rate text into structured data, reducing the risk of human involvement and significantly improving the efficiency and accuracy of data processing, adapting to the high-frequency and high-precision processing needs of international logistics industry for freight rate information.

[0060] In one embodiment, the step of using a large language model to extract initial key information and note information from sea freight rate text includes:

[0061] The first prompt instruction is obtained and input into the large language model to obtain sea transportation basic information, including the port of departure, the destination port, and the ship name and voyage.

[0062] According to the sea transportation basic information, a second prompt instruction is determined and input into the large language model to obtain container type price information, which includes the container type price of each port, taking the sea transportation basic information and the container type price information as initial key information.

[0063] According to the sea transportation basic information, a third prompt instruction is determined and input into the large language model to obtain remark information, including general remarks for each port and special remarks for specific ports.

[0064] Among them, the sea transportation basic information is information including the port of departure, the destination port, and the ship name and voyage, which are the core data for sea transportation transactions and transportation. The port of departure refers to the port where the goods are loaded, the destination port is the port where the goods are unloaded, and the ship name and voyage are the specific ship and its voyage number. The container type price information refers to the transportation cost of different types of containers between specific routes and ports, which is usually divided according to the specifications and purposes of the containers, such as standard containers, refrigerated containers, etc. The remark information is an additional explanation of the sea transportation basic information and the container type price information, which may include general remarks applicable to all ports and special remarks for specific ports, such as transportation restrictions, special requirements, etc., which helps to perfect and clarify the information.

[0065] Specifically, first, the first prompt instruction is received, which can be input into the large language model through a user interface or API call. The large language model analyzes the input data and extracts sea transportation basic information according to the instruction. The first prompt instruction can include relevant concept explanations, such as "ETD date must exist, format is generally `%m.%d`, written together with ship name and voyage or route code, if there is no ship name and voyage or route code, it will be directly marked with ETD near the date"; "identify all ship names and voyages vessel / vol (not unique) and their corresponding ETD dates (if the year information is missing, complete the current date with the closest year) and route codes route_code (usually 3 digits, may be empty, pay attention to distinguish from port three-digit code), if there is no ship name and voyage, then default to build a record with an empty string for ship name and voyage and today's ETD"; "supplement knowledge, commonly used domestic port cities (and abbreviations): Ningbo (NB), Shanghai (SH), Shekou (SK), Yantian (YT), Dachang Bay (DCB), Nansha (NS), Huangpu (HP), etc.". Further, the first prompt instruction can construct null content, such as default value today / empty character, etc.

[0066] After obtaining the basic shipping information, the second prompt instruction is generated by analyzing these information, which may involve constructing a query string or statement to explicitly ask for the container type price information. This instruction is then input into the large language model again, and the instruction contains the required context to obtain the relevant container type price information. Exemplarily, the second prompt instruction can be "When the container type is not mentioned in the full text, "price1 / price2 / price3" represents "20GP / 40GP / 40HQ" respectively; if there is no "price3-40HQ", 40HQ is equal to the price of 40GP by default; if there is no "price2-40GP", 40GP is equal to the price of 40HQ by default"; when `+` appears in the original information price, it means that the price of this group needs to be calculated based on the previous content"; "ENS / PCS and other additional fee information do not need to be included in the container type price"

[0067] Similarly, according to the previous basic shipping information, the third prompt instruction is generated to obtain the relevant note information for each port. This process also calls the large language model to ensure that the instruction is accurate and has the necessary context to facilitate the model to return valid note information.

[0068] In this embodiment, by inputting the first prompt instruction into the large language model and obtaining the basic shipping information, the core data can be automatically extracted, the speed and accuracy of information extraction can be improved, the errors of manual input can be reduced, and the overall business efficiency can be improved. The second prompt instruction is automatically generated to ensure the full use of basic shipping information, reduce the risk of information silos, and quickly and accurately combine these data with basic information. Obtaining note information helps to provide additional context to ensure that all relevant factors are considered when determining the shipping rate or transportation arrangement. By automatically generating instructions, not only the efficiency is improved, but also the accuracy and completeness of the note information are ensured.

[0069] In one embodiment, the step of modifying the initial key information to obtain the modified key information includes:

[0070] Obtaining the shipping information correction data;

[0071] Fuzzy matching the initial key information with the shipping information correction data to obtain a first matching result, and modifying the spelling error information in the initial key information and eliminating the completely wrong information in the initial key information according to the first matching result to obtain the modified key information.

[0072] The sea transportation information correction data refers to a set of standard information related to sea transportation, used to verify and correct the accuracy of the initial key information. These data usually include the standard spelling of the port of departure, the port of destination, the ship name and voyage, etc., as the basis for comparison. The first matching result is the result set of the preliminary fuzzy matching, showing the matching degree between the initial key information and the sea transportation information correction data, which may contain correct and partial matching information. The corrected key information is the accurate key information set obtained after correction and correction, which contains information corrected for spelling errors and excludes incorrect data to ensure the validity and consistency of the data.

[0073] Specifically, first, sea transportation information correction data can be extracted from predefined data sources such as databases, files or APIs. After obtaining the correction data, use fuzzy matching algorithms such as Levenshtein distance, Jaccard similarity, etc. to compare the initial key information with the sea transportation information correction data. Compare each initial key information with the corresponding item in the correction data to generate the first matching result and determine which information is similar, has spelling errors or is completely wrong. Based on the first matching result, the initial key information will be processed. For information that matches successfully but has spelling errors, use the standard item in the correction data to replace it; and for completely unmatched information, it will be excluded. Finally, generate the corrected key information to form a clean data set.

[0074] In this embodiment, fuzzy matching can effectively identify and correct spelling errors in the initial key information. By comparing the information with standard data, the accuracy of the data can be improved, and the problems caused by manual input errors can be reduced. This process reduces the need for manual review and improves processing efficiency. The corrected key information after correction will significantly improve the accuracy and consistency of the data, ensuring that the data used in subsequent processing, analysis and decision-making is reliable.

[0075] In one embodiment, by using a large language model and a pre-acquired port five-character code table, the non-standard port name in the corrected key information is converted into a standard port name to obtain the converted key information, including:

[0076] The port name in the corrected key information is matched with the port name in the port five-character code table to obtain a second matching result, and the non-standard port name in the corrected key information is replaced with the standard port name in the port five-character code table according to the second matching result to obtain the converted key information.

[0077] The second matching result is a result set obtained by matching the port name in the corrected key information with the port name in the port five-character code table, containing the matching successful port name and the unmatched name, indicating which name needs to be replaced or corrected. The converted key information is a key information set in which the port name has been replaced by the standard name in the port five-character code table after standardization processing, ensuring the consistency and standardization of the information.

[0078] Specifically, first, the port name in the corrected key information is compared with the port name in the port five-character code table. This process fuzzy matches the similarity between the two, traverses each port name in the corrected key information, and finds the corresponding standard name in the port five-character code table. After the matching process is completed, the second matching result is generated, recording the matching state of each port name. Among them, for the successfully matched name, the standard name is recorded, and for the unmatched name, it is marked as an item that needs further processing. After obtaining the second matching result, all non-standard port names in the corrected key information are replaced by the standard port name in the port five-character code table. By traversing the corrected key information, the successfully matched standard name is filled into the corresponding position to form the converted key information.

[0079] In this embodiment, by matching the corrected key information with the port five-character code table, potential naming inconsistencies can be identified and corrected, which helps to eliminate data confusion caused by spelling errors or the use of different names. Replacing the non-standard port name in the corrected key information with the standard name not only improves the consistency and standardization of the data, but also simplifies subsequent data processing and analysis.

[0080] In one embodiment, the step of adjusting each port corresponding to the voyage name in the converted key information according to the route correspondence table to obtain the effective key information comprises:

[0081] For each voyage name in the converted key information, the voyage name is matched with the route correspondence table to determine each port of call corresponding to the voyage name, and the initial port corresponding to the voyage name that is inconsistent with each port of call is removed to obtain the effective key information.

[0082] The voyage name refers to the ship name and voyage information of a specific shipping route, usually composed of a ship name and a voyage number, used to uniquely identify a specific shipping path.

[0083] Specifically, first, each voyage in the converted key information is traversed, and matching is performed according to the route correspondence table. The matching process can be implemented through a lookup operation, which looks up the corresponding port of call in the route correspondence table according to the voyage. For each successfully matched voyage, the corresponding port of call information is extracted. Next, the relationship between the initial port, i.e., the port corresponding to the voyage in the converted key information, and the port of call is compared. By traversing the initial port, it is checked whether each port matches any port in the set of ports of call. For ports that do not match, they are removed, thereby obtaining valid key information.

[0084] In this embodiment, by matching the voyage in the converted key information with the route correspondence table, the complete route of each voyage can be identified, ensuring that each voyage accurately reflects its actual transportation path. Extracting the port of call information ensures the use of accurate shipping path data in subsequent processing. Removing the initial port that does not match the port of call helps to ensure the accuracy and consistency of the valid key information. After screening, a key information set that truly reflects the actual shipping situation can be generated, avoiding decision-making errors caused by false data.

[0085] In one embodiment, the step of verifying the box type price of the same named port in each entity relationship using a large language model includes:

[0086] Using a large language model, the consistency of the box type price of the same named port in each entity relationship is verified. If there is a difference in the box type price of the same named port, the box type price with the difference is screened and corrected according to the preset rules.

[0087] The preset rules are rules used to guide and regulate how to handle price differences during data processing. These rules can be fixed logic or dynamically generated based on data analysis results.

[0088] Specifically, first, the consistency of the box type price of the same named port in each entity relationship is verified using a large language model. The price information of the same named port is extracted, and through text analysis and comparison, it is identified which same named ports have different box type prices. After discovering the price difference, it will be processed according to the preset rules. Illustratively, all prices with differences are recorded, and reasonable prices are selected according to rules such as priority, timestamp, source, etc. For prices that do not meet the preset rules, they will be corrected and replaced with the standard price after screening.

[0089] In this embodiment, by using a large language model to verify the consistency of the box type prices of ports with the same name, potential data errors or inconsistencies can be quickly identified, improving the quality and accuracy of the data and avoiding decision-making errors or business problems caused by incorrect data. After discovering price differences, using pre-set rules for screening and correction can ensure the consistency and standardization of data processing. By clearly defining the processing rules, complex data situations can be handled more effectively, reducing manual intervention and improving processing efficiency.

[0090] In one embodiment, the step of generating a sea freight rate table according to each entity relationship includes:

[0091] Generating a sub-table corresponding to each entity relationship and mapping each sub-table through Cartesian product to obtain a sea freight rate table.

[0092] Wherein, the sub-table refers to a smaller table extracted from the main data set, which contains information of a specific entity relationship. For example, `Port x Price Table`, `Port x Voyage Table`, `Port x Remark Table`, `Price x Voyage Table`, `Port x Five-Character Code Table`. Cartesian product is a mathematical operation that combines elements from two or more sets. For each element in each set, Cartesian product generates a new combination. For example, if there are two sets A and B, Cartesian product A x B will generate all possible element combinations.

[0093] Specifically, relevant information is extracted from entity relationships and a sub-table is generated for each entity relationship. This can be achieved through data query and filtering functions to extract data from the main data set according to conditions and organize it into small tables. For example, a sub-table containing all ports and their corresponding prices is generated from port and price data. After all sub-tables are generated, they are mapped using the Cartesian product method. All rows in each sub-table are traversed and combined with relevant data from other sub-tables to form all possible combinations, which can be achieved through nested loops or using the join operation in the database.

[0094] In this embodiment, by generating sub-tables, complex raw data can be broken down into smaller, more manageable parts. Mapping each sub-table using Cartesian product can produce a comprehensive and detailed sea freight rate table, ensuring that all relevant information is fully integrated.

[0095] The following describes a sea freight rate text recognition device based on a large language model provided by the embodiments of the present application. The sea freight rate text recognition device based on a large language model described below can be mutually corresponding to the sea freight rate text recognition method based on a large language model described above. As shown in Figure 2 The present application provides a sea freight rate text recognition device based on a large language model, which comprises:

[0096] The sea freight rate text acquisition module 201 is configured to acquire a sea freight rate text of international logistics, and extract initial key information and remark information from the sea freight rate text by using a large language model, wherein the initial key information includes a port of departure, a destination port, a ship name and voyage, and a container type price.

[0097] The port name standardization module 202 is configured to correct the initial key information to obtain corrected key information, and convert non-standard port names in the corrected key information into standard port names by using a port five-code table obtained in advance by using the large language model, to obtain converted key information.

[0098] The target key information determination module 203 is configured to acquire a route correspondence table for indicating a correspondence between a ship name and voyage and a port, adjust the ports corresponding to each ship name and voyage in the converted key information according to the route correspondence table to obtain valid key information, and supplement the valid key information with the remark information to obtain target key information.

[0099] The sea freight rate table generation module 204 is configured to generate a plurality of entity correspondence relationships by using the large language model according to the target key information, and generate a sea freight rate table according to each entity relationship after verifying the container type prices of ports with the same name in each entity relationship by using the large language model.

[0100] In an embodiment, the sea freight rate text acquisition module 201 includes:

[0101] The sea freight basic information acquisition unit is configured to acquire a first prompt instruction, and input the first prompt instruction into the large language model to obtain sea freight basic information, wherein the sea freight basic information includes a port of departure, a destination port, and a ship name and voyage.

[0102] The initial key information determination unit is configured to determine a second prompt instruction according to the sea freight basic information, and input the second prompt instruction into the large language model to obtain container type price information, so as to take the sea freight basic information and the container type price information as initial key information, wherein the container type price information includes a container type price of each port.

[0103] The remark information acquisition unit is configured to determine a third prompt instruction according to the sea freight basic information, and input the third prompt instruction into the large language model to obtain remark information, wherein the remark information includes general remarks of each port and special remarks of a specific port.

[0104] In an embodiment, the port name standardization module 202 includes:

[0105] The sea freight information proofreading data acquisition unit is configured to acquire sea freight information proofreading data.

[0106] The corrected key information determination unit is configured to perform fuzzy matching between the initial key information and the sea transportation information collation data to obtain a first matching result, and correct spelling errors in the initial key information and eliminate completely incorrect information in the initial key information according to the first matching result, to obtain corrected key information.

[0107] In one embodiment, the port name standardization module 202 includes:

[0108] The port name standardization unit is configured to perform matching between the port name in the corrected key information and the port name in the port five-character code table to obtain a second matching result, and replace the non-standard port name in the corrected key information with the standard port name in the port five-character code table according to the second matching result, to obtain converted key information.

[0109] In one embodiment, the target key information determination module 203 includes:

[0110] The valid key information determination unit is configured to, for each voyage in the converted key information, perform matching between the voyage and the route corresponding table to determine each transit port corresponding to the voyage, and eliminate the port in the initial port corresponding to the voyage that is inconsistent with each transit port, to obtain valid key information.

[0111] In one embodiment, the sea transportation freight table generation module 204 includes:

[0112] The container type price verification unit is configured to use a large language model to verify the consistency of the container type prices of the same-named ports in each entity relationship, and if there is a difference in the container type prices of the same-named ports, perform screening and correction on the container type prices with differences according to a preset rule.

[0113] In one embodiment, the sea transportation freight table generation module 204 includes:

[0114] The sea transportation freight table generation unit is configured to generate a sub-table corresponding to each entity relationship, and map each sub-table through a Cartesian product to obtain a sea transportation freight table.

[0115] In one embodiment, the present application also provides a storage medium having computer readable instructions stored therein, and the computer readable instructions are executed by one or more processors to cause the one or more processors to perform the steps of the sea transportation freight text recognition method based on a large language model as described in any of the above embodiments.

[0116] In an embodiment, the present application also provides a computer device having computer readable instructions stored therein, which, when executed by one or more processors, cause the one or more processors to perform the steps of the method for sea freight rate text recognition based on large language model according to any of the above embodiments.

[0117] As shown schematically, Figure 3 , Figure 3 Fig. 6 is a schematic diagram of an internal structure of a computer device according to an embodiment of the present application. The computer device 300 can be provided as a server. As shown in Figure 3 , the computer device 300 includes a processing assembly 302, which further includes one or more processors, and a memory resource represented by a memory 301 for storing instructions, such as application programs, executable by the processing assembly 302. The application programs stored in the memory 301 can include one or more than one module each corresponding to a set of instructions. In addition, the processing assembly 302 is configured to execute the instructions to perform the method for sea freight rate text recognition based on large language model according to any of the above embodiments.

[0118] The computer device 300 can further include a power supply assembly 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input output (I / O) interface 305. The computer device 300 can operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0119] Those skilled in the art can understand that Figure 3 the structure shown in the above embodiment is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. Specifically, the computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] Finally, it should be noted that the terms "first" and "second", and the like, herein do not denote any order, quantity, combination or importance, but are used to identify one element from another, and do not imply that the specific identities thereof are essential or that the identities are chronological or related in their occurrence. Also, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the recited element. Also, "a", "an", "the", and "said" are used to refer to one or more than one (i.e., to "at least one") of the referenced elements, unless otherwise specified. A plurality also means two or more, such as 2, 3, 5, or 8, etc. "And / or" includes any and all combinations of one or more of the associated listed items.

[0121] The various embodiments described in this specification are presented by way of example, and are not intended to limit the scope of the application. Each embodiment is presented in a way that emphasizes the differences between the embodiments and the other embodiments, but the embodiments can be combined according to the needs of the user, and the same or similar parts are cross-referenced.

[0122] The above description of disclosed embodiments provides enabling disclosure sufficient for one of ordinary skill in the art to implement or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for recognizing ocean freight rate text based on a large language model, characterized in that, The method includes: Obtain the ocean freight rate text for international logistics, and use a large language model to extract initial key information and remarks from the ocean freight rate text. The initial key information includes the port of origin, port of destination, vessel name and voyage number, and container type price. The initial key information is corrected to obtain corrected key information. Then, using the large language model and the pre-acquired port five-character code table, the non-standard port names in the corrected key information are converted into standard port names to obtain converted key information. Obtain a route correspondence table that represents the correspondence between ship names and voyages and ports. According to the route correspondence table, adjust the ports corresponding to each voyage name and voyage in the converted key information to obtain effective key information. Then, supplement the effective key information with the remarks information to obtain the target key information. Based on the target key information, multiple entity correspondences are generated using the large language model. After verifying the container prices of the same port in each entity relationship using the large language model, a sea freight rate table is generated according to each entity relationship.

2. The method for recognizing ocean freight rate text based on a large language model according to claim 1, characterized in that, The steps of extracting initial key information and remarks from the ocean freight rate text using a large language model include: Obtain the first prompt instruction and input the first prompt instruction into the large language model to obtain basic maritime information, which includes the port of departure, port of destination, and vessel name and voyage number. Based on the basic shipping information, a second prompt instruction is determined and input into the large language model to obtain container price information. The basic shipping information and the container price information are used as the initial key information, and the container price information includes the container price for each port. Based on the basic maritime information, a third prompt instruction is determined and input into the large language model to obtain the remarks information, which includes general remarks for various ports and special remarks for specific ports.

3. The method for recognizing ocean freight rate text based on a large language model according to claim 1, characterized in that, The step of correcting the initial key information to obtain corrected key information includes: Obtain maritime information and verify data; The initial key information is fuzzily matched with the maritime information verification data to obtain a first matching result. Based on the first matching result, the spelling errors in the initial key information are corrected, and the completely erroneous information in the initial key information is removed to obtain the corrected key information.

4. The method for recognizing ocean freight rate text based on a large language model according to claim 1, characterized in that, The step of converting non-standard port names in the corrected key information into standard port names using the large language model and a pre-acquired port five-character code table to obtain the converted key information includes: The port names in the corrected key information are matched with the port names in the port five-letter code table to obtain a second matching result. Based on the second matching result, the non-standard port names in the corrected key information are replaced with standard port names in the port five-letter code table to obtain the converted key information.

5. The method for recognizing ocean freight rate text based on a large language model according to claim 1, characterized in that, The step of adjusting the ports corresponding to each voyage name and voyage in the converted key information according to the route correspondence table to obtain valid key information includes: For each voyage name and voyage in the converted key information, the voyage name and voyage name are matched with the route correspondence table to determine the ports along the way corresponding to the voyage name and voyage name. Ports that are inconsistent with the ports along the way in the initial port corresponding to the voyage name and voyage name are removed to obtain the effective key information.

6. The method for recognizing ocean freight rate text based on a large language model according to claim 1, characterized in that, The step of verifying the container prices of ports with the same name in each of the entity relationships using the large language model includes: The large language model is used to verify the consistency of container prices for ports with the same name in each entity relationship. If there are differences in container prices for ports with the same name, the differences in container prices are filtered and corrected according to preset rules.

7. The method for recognizing ocean freight rate text based on a large language model according to any one of claims 1 to 6, characterized in that, The step of generating the ocean freight rate table according to the relationships between the entities includes: A sub-table corresponding to each entity relationship is generated, and each sub-table is mapped using a Cartesian product to obtain the ocean freight rate table.

8. A text recognition device for ocean freight rates based on a large language model, characterized in that, The device includes: The ocean freight rate text acquisition module is used to acquire ocean freight rate texts for international logistics and to extract initial key information and remarks from the ocean freight rate texts using a large language model. The initial key information includes the port of origin, port of destination, vessel name and voyage number, and container type price. The port name standardization module is used to correct the initial key information to obtain corrected key information, and through the large language model, using the pre-acquired port five-character code table, convert the non-standard port names in the corrected key information into standard port names to obtain converted key information. The target key information determination module is used to obtain a route correspondence table that represents the correspondence between ship names and voyages and ports, adjust the ports corresponding to each voyage name and voyage in the converted key information according to the route correspondence table, obtain effective key information, and supplement the effective key information with the remarks information to obtain target key information. The ocean freight rate table generation module is used to generate multiple entity correspondences based on the target key information using the large language model, and after verifying the container prices of the same port in each entity relationship using the large language model, generate an ocean freight rate table according to each entity relationship.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the ocean freight rate text recognition method based on any one of claims 1 to 7.

10. A computer device, characterized in that, include: One or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the ocean freight rate text recognition method based on any one of claims 1 to 7.

Images