Marine transportation price text recognition method based on large language model

Abstract

According to the marine transportation price text recognition method based on the large language model, a marine transportation price text is acquired, and initial key information and remark information are extracted from the marine transportation price text by adopting the large language model; after the initial key information is corrected, non-standard port names in the corrected key information are converted into standard port names through a big language model by using a port five-character code table, and converted key information is obtained; adjusting a port corresponding to each flight name and flight number in the converted key information according to the obtained flight line corresponding table to obtain effective key information, and supplementing the effective key information by adopting remark information to obtain target key information; and according to the target key information, adopting a large language model to generate a plurality of entity corresponding relationships, adopting the large language model to verify the box type price of the homonymous port in each entity relationship, and generating a sea transportation price table according to each entity relationship. According to the method, the freight rate information extraction efficiency of the unstructured marine freight rate text is improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a method for recognizing shipping freight rate text based on a large language model. Background Art

[0002] Against the backdrop of the rapid development of global trade, the international logistics industry is shouldering an increasingly heavy demand for cargo transportation, and the management and identification of freight information plays an important role in this. Especially in the field of shipping, changes in factors such as different regions, routes, and cargo types have led to a significant increase in the complexity of freight information. However, since freight rates usually exist in the form of unstructured text and involve a large amount of natural language descriptions, their parsing and processing has become a very challenging task.

[0003] Existing technologies rely on middle and senior personnel with rich industry experience to manually understand and convert freight rate texts to parse complex natural language content into structured information. This method not only consumes a lot of manpower, but the conversion efficiency and accuracy are easily affected by the professional level of the operators, and it is difficult to adapt to large-scale, high-frequency data processing needs. Under the current trend of digital transformation, the unstructured text form of shipping freight rates is difficult to be programmatically parsed into structured data that can be directly used by the system, resulting in the need to rely on manual analysis, making the efficiency of shipping freight rate text recognition low. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical deficiencies, especially the technical defect of low efficiency of shipping freight rate text recognition in the prior art.

[0005] In a first aspect, the present application provides a method for recognizing shipping freight rates text based on a large language model, the method comprising:

[0006] Obtain the text of international logistics ocean freight rates, and use the large language model to extract the initial key information and remarks from the text of ocean freight rates. The initial key information includes the port of departure, port of destination, ship name, voyage number, and container type price;

[0007] The initial key information is corrected to obtain corrected key information, and the non-standard port names in the corrected key information are converted into standard port names by using a large language model and a pre-acquired port five-character code table to obtain converted key information;

[0008] Obtain a route correspondence table for indicating the correspondence between a ship name and a voyage and a port, adjust the port corresponding to each voyage in the converted key information according to the route correspondence table, obtain valid key information, and supplement the valid key information with remark information to obtain target key information;

[0009] According to the target key information, a large language model is used to generate multiple entity correspondences. After verifying the container prices of the ports with the same name in each entity relationship, the shipping freight rate table is generated according to each entity relationship.

[0010] In one embodiment, the step of extracting initial key information and remark information from the shipping freight rate text using a large language model includes:

[0011] Obtain a first prompt instruction, and input the first prompt instruction into a large language model to obtain basic shipping information, which includes a departure port, a destination port, and a ship name and voyage number;

[0012] According to the basic information of shipping, determine the second prompt instruction, and input the second prompt instruction into the large language model to obtain the container price information, so as to use the basic information of shipping and the container price information as the initial key information, and the container price information includes the container price of each port;

[0013] According to the basic information of shipping, the third prompt instruction is determined, and the third prompt instruction is input into the large language model to obtain the remark information, which includes the general remark of each port and the special remark of the specific port.

[0014] In one embodiment, the step of correcting the initial key information to obtain the corrected key information includes:

[0015] Obtain proofreading data on shipping information;

[0016] The initial key information is fuzzily matched with the shipping information proofreading data to obtain a first matching result, and based on the first matching result, misspelled information in the initial key information is modified, and completely wrong information in the initial key information is eliminated to obtain corrected key information.

[0017] In one embodiment, the step of converting the non-standard port name in the corrected key information into a standard port name by using a large language model and a pre-acquired port five-character code table to obtain the converted key information includes:

[0018] The port name in the corrected key information is matched with the port name in the port five-character code table to obtain a second matching result, and based on the second matching result, the non-standard port name in the corrected key information is replaced with the standard port name in the port five-character code table to obtain the converted key information.

[0019] In one embodiment, the step of adjusting the port corresponding to each voyage name in the converted key information according to the route correspondence table to obtain valid key information includes:

[0020] For each voyage in the converted key information, the voyage is matched with the route correspondence table to determine the various ports corresponding to the voyage, and the ports in the initial ports corresponding to the voyage that are inconsistent with the various ports are eliminated to obtain valid key information.

[0021] In one embodiment, the step of using a large language model to verify the container price of ports with the same name in each entity relationship includes:

[0022] A large language model is used to verify the consistency of the container prices of ports with the same name in each entity relationship. If there are differences in the container prices of ports with the same name, the different container prices are screened and corrected according to the preset rules.

[0023] In one embodiment, the step of generating a shipping rate table according to each entity relationship includes:

[0024] Generate a subtable corresponding to each entity relationship, and map each subtable through the Cartesian product method to obtain the ocean freight rate table.

[0025] In a second aspect, the present application provides a device for recognizing shipping freight rates text based on a large language model, the device comprising:

[0026] The shipping freight rate text acquisition module is used to obtain the shipping freight rate text of international logistics, and use the large language model to extract the initial key information and remark information from the shipping freight rate text. The initial key information includes the departure port, destination port, ship name, voyage number and container type price;

[0027] The port name standardization module is used to correct the initial key information to obtain the corrected key information, and convert the non-standard port name in the corrected key information into a standard port name by using the pre-acquired port five-character code table through a large language model to obtain the converted key information;

[0028] The target key information determination module is used to obtain a route correspondence table for indicating the correspondence between the ship name and the port, and adjust the port corresponding to each ship name and the port in the converted key information according to the route correspondence table to obtain the valid key information, and supplement the valid key information with the remark information to obtain the target key information;

[0029] The shipping freight rate table generation module is used to generate multiple entity correspondences based on the target key information using a large language model, and after verifying the container prices of the same-name ports in each entity relationship using the large language model, generate a shipping freight rate table according to each entity relationship.

[0030] In a third aspect, the present application provides a storage medium: the storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the shipping freight rate text recognition method based on a large language model as described in any of the above embodiments.

[0031] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0032] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the steps of the method for recognizing shipping freight rates text based on a large language model in any of the above embodiments are performed.

[0033] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0034] In the ocean freight rate text recognition method based on a large language model provided in the present application, the ocean freight rate text is obtained and the initial key information and remark information are automatically extracted using a large language model, and the unstructured text content is initially converted into structured data; the non-standard port names in the initial key information are corrected and standardized using a large language model and a pre-acquired port five-character code table, and the port names are unified into standardized expressions, which not only reduces the risk of human misjudgment, but also improves the uniformity and consistency of information; after obtaining the correspondence table between ship names and voyages and ports, the table is used to adjust the port corresponding to each ship name and voyage in the converted key information, and the key information that conforms to the business logic is screened out. In addition, the remark information is used to supplement the effective key information to make the information more complete; based on the effective key information, the corresponding relationship between multiple entities is generated using a large language model, which effectively reduces errors caused by complex relationship intersections, and ensures that the data is accurate by verifying the box type prices of ports with the same name; finally, the ocean freight rate table is generated using the verified entity relationship, so that the originally unstructured freight rate information is converted into structured table data. Based on this, this method uses the natural language parsing capabilities of a large language model, combined with auxiliary information such as the port five-character code table and the route correspondence table, to efficiently convert the shipping freight rate text that originally relied on human recognition into structured data, reducing the risk of human participation and significantly improving the efficiency and accuracy of data processing, adapting to the international logistics industry's high-frequency and high-precision processing needs for freight rate information. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 A flow chart of a method for recognizing shipping freight rates text based on a large language model provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of the structure of a shipping freight rate text recognition device based on a large language model provided in an embodiment of the present application;

[0038] Figure 3 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] This application provides a method for recognizing shipping freight rates based on a large language model. The following embodiments are described by taking the method applied to a video platform as an example. It can be understood that the video platform can be any device with data processing functions, including but not limited to a single server and a server cluster. Figure 1 As shown, the method may include the following steps:

[0041] S101: Obtain the shipping freight rate text of international logistics, and use the large language model to extract the initial key information and remark information from the shipping freight rate text. The initial key information includes the port of departure, port of destination, ship name, voyage number and container type price.

[0042] Among them, the ocean freight text of international logistics refers to unstructured text data containing international ocean freight service price information, usually including information such as the port of departure and destination, ship name and voyage, cargo type, container type and freight rate. These texts can come from multiple sources, such as shipping companies, freight forwarders, etc., and are usually in the form of natural language text. The large language model (LLM) refers to a deep learning model with powerful natural language processing capabilities that can extract, analyze and structure information from unstructured text. The initial key information is the basic business elements extracted from the text, including the port of departure, port of destination, ship name and voyage, and container type price. The remark information is additional information related to the key information, usually including the applicable conditions, restrictions or supplementary instructions of the freight rate, which helps to improve and accurately describe and analyze the freight rate.

[0043] In this step, the shipping freight rate text input or uploaded by the user can be received, and the obtained freight rate text can be input into LLM for information extraction. Exemplarily, LLM first performs word segmentation, part-of-speech tagging and syntactic parsing on the text, and then locates and extracts fields such as the port of departure, port of destination, ship name, voyage and container type price, and outputs the content of each field in a structured form. The large language model can adopt the COT (Chain of Thought) reasoning method, that is, process the text step by step to extract key information and additional notes from it. In this reasoning process, the text information can be effectively layered to ensure the accurate extraction of key information and notes.

[0044] It can be understood that the application of large language models avoids the reliance on grammar of traditional rule-based text parsing methods, making the model more flexible and accurate in processing diverse and complex natural language descriptions. LLM can accurately identify and extract key information even if there are differences in text format and content.

[0045] Remarks often contain special instructions for freight rates, which are crucial for business analysis. LLM's COT method can extract key information and supplementary notes layer by layer, ensuring that key information is complete and notes are clear, making the data more accurate in subsequent verification, analysis and generation processes. At the same time, this automated method reduces the risk of errors caused by human intervention and improves processing efficiency.

[0046] S102: Correct the initial key information to obtain corrected key information, and convert the non-standard port names in the corrected key information into standard port names by using the pre-acquired port five-character code table through a large language model to obtain converted key information.

[0047] Among them, the corrected key information refers to the key information obtained after inspection, proofreading, and possible manual or automatic correction. The port five-character code table is a standardized mapping table containing port names, which maps the names of ports around the world to a unified five-character code, including the Chinese name of the port, the English name, and the standard five-character code. The table is used to identify and unify different port names to make them conform to industry standards. The converted key information refers to the final key information that replaces the non-standard port name in the corrected key information with the standardized port name after using the port five-character code table.

[0048] In this step, you can automatically scan the initial key information to correct possible spelling or format errors by writing a check function or enabling a large language model. LLM can check the integrity and format consistency of the key information, and automatically complete and correct obvious spelling errors or missing information. Then, by calling the port five-character code table, the non-standard port names in the corrected key information are matched and replaced with five-character codes or standard port names.

[0049] It is understandable that since there may be spelling errors, inconsistent formats and other issues in the freight rate text, correcting the initial key information through automated means can effectively improve accuracy. Compared with manual corrections, computer equipment can quickly process large amounts of text, reducing delays and errors caused by manual operations. Port name standardization is very important for accurately matching port information, performing freight rate statistics and data analysis. Freight rate texts from different sources may use different port descriptions, and even contain abbreviations or spelling errors. The use of standard port names makes the information highly consistent, eliminating confusion and errors caused by inconsistent names.

[0050] S103: Obtain a route correspondence table for indicating the correspondence between the ship name and the voyage and the port, adjust the port corresponding to each voyage in the converted key information according to the route correspondence table, obtain valid key information, and supplement the valid key information with the remark information to obtain the target key information.

[0051] Among them, the route correspondence table is a table that records the correspondence between the ship's voyage and the ports it passes through, including information such as the voyage, port, and arrival / departure time. This table is used to identify the ports that the ship stops at in sequence in a specific voyage to ensure the accuracy of the voyage data. The effective key information is the key information that matches the actual voyage and port according to the route correspondence table, excluding invalid port information to ensure that the data conforms to the actual situation of the voyage. The target key information is the final key information formed after the remark information is supplemented, which conforms to the actual voyage and contains necessary additional instructions, forming complete and accurate freight information.

[0052] In this step, the route correspondence table can be automatically obtained and updated through API calls, scheduled tasks or database connections. For example, it can be set to periodically extract the latest route data from the specified source and store it in the local database for subsequent queries. Filter and adjust the converted key information according to the route correspondence table. After that, automatically associate the remark information with the valid key information. For example, by field matching or ID association, the remark information is attached to the corresponding valid key information item to form a more complete target key information, ensuring that the remark information is output synchronously when the freight rate table is generated.

[0053] It is understandable that since routes may change or be updated over time, obtaining a route correspondence table can ensure the accuracy and timeliness of information. Adjusting the port correspondence can avoid interference from invalid data. By screening out valid ports that match the actual routes, the accuracy of key information can be ensured. In this way, the data quality is improved, the time cost of manual screening is avoided, and the freight rate information is ensured to be consistent with the actual routes, which helps to improve the accuracy of the freight rate table. Remarks information usually provides special applicable conditions or restrictions. Supplementing this information can make the target key information more comprehensive. Associating remark information can improve accuracy, avoid missing key applicable conditions, and thus reduce misunderstandings and incorrect use.

[0054] S104: Based on the target key information, a large language model is used to generate multiple entity correspondences, and after the large language model is used to verify the container prices of the ports with the same name in each entity relationship, a sea freight rate table is generated according to each entity relationship.

[0055] Among them, the entity correspondence is the relationship between different entities in data processing. For example, the relationship between the port and its corresponding container type price, the connection between the voyage and the port, etc. The container type price refers to the freight price of different types of containers between a specific voyage and port. The ocean freight rate table is a structured data table generated based on the target key information and the entity correspondence, which contains the voyage, departure port, destination port, container type and corresponding freight rate information.

[0056] In this step, the target key information is analyzed through the large language model to automatically identify and extract the relationship between different entities, such as the shipping route between the port of departure and the port of destination, the box type price of the port with the same name, etc. The system will use natural language processing (NLP) technology to convert the key information into a format that can be understood by the machine, forming a relationship map or association matrix for subsequent analysis. After that, the large language model is used to cross-validate the box type prices of the ports with the same name for the generated entity relationships. For example, by finding the price data in the database that matches the target key information, the box type price of each port is ensured to be consistent and accurate.

[0057] Once the verification is completed, the information can be integrated into a structured table through the data processing module according to the entity correspondence and the verification results. For example, a data visualization tool or table processing library is used to generate a shipping freight rate table containing information such as voyage, departure port, destination port, container type and freight rate, and finally output it to the required file format, such as CSV, Excel, etc.

[0058] It can be understood that the entity correspondence generated by the large language model can effectively sort out and organize the data structure and understand the intrinsic connection between different data. Verifying the box prices of ports with the same name ensures the reliability of freight rate data. Since box prices may be affected by factors such as market fluctuations and voyage differences, the automatic verification mechanism can quickly identify and solve potential problems. The final generated ocean freight rate table makes the data presentation clearer and more systematic, which is convenient for subsequent query, statistics and analysis. In addition, the automated process of table output has greatly improved work efficiency and reduced the cost and risk of manual operation.

[0059] In the above embodiment, the unstructured text content is initially converted into structured data by obtaining the shipping freight text and automatically extracting the initial key information and remark information using the large language model; the non-standard port names in the initial key information are corrected and standardized using the large language model and the pre-acquired port five-character code table, and the port names are unified into standardized expressions, which not only reduces the risk of human misjudgment, but also improves the uniformity and consistency of information; after obtaining the correspondence table between ship names and voyages and ports, the table is used to adjust the port corresponding to each ship name and voyage in the converted key information, and the key information that conforms to the business logic is screened out. In addition, the remark information is used to supplement the valid key information to make the information more complete; based on the valid key information, the large language model is used to generate the corresponding relationship between multiple entities, which effectively reduces the errors caused by the intersection of complex relationships, and ensures the accuracy of the data by verifying the box type prices of the ports with the same name; finally, the shipping freight table is generated using the verified entity relationship, so that the original unstructured freight information is converted into structured table data. Based on this, this method uses the natural language parsing capabilities of a large language model, combined with auxiliary information such as the port five-character code table and the route correspondence table, to efficiently convert the shipping freight rate text that originally relied on human recognition into structured data, reducing the risk of human participation and significantly improving the efficiency and accuracy of data processing, adapting to the international logistics industry's high-frequency and high-precision processing needs for freight rate information.

[0060] In one embodiment, the step of extracting initial key information and remark information from the shipping rate text using a large language model includes:

[0061] Obtain a first prompt instruction, and input the first prompt instruction into a large language model to obtain basic shipping information, which includes a departure port, a destination port, and a ship name and voyage number;

[0062] According to the basic information of shipping, determine the second prompt instruction, and input the second prompt instruction into the large language model to obtain the container price information, so as to use the basic information of shipping and the container price information as the initial key information, and the container price information includes the container price of each port;

[0063] According to the basic information of shipping, the third prompt instruction is determined, and the third prompt instruction is input into the large language model to obtain the remark information, which includes the general remark of each port and the special remark of the specific port.

[0064] Among them, the basic information of shipping includes the port of departure, the port of destination, and the ship name and voyage number, which are the core data for shipping transactions and transportation. The port of departure refers to the port where the goods are loaded, the port of destination is the port where the goods are unloaded, and the ship name and voyage number refers to the specific ship and its voyage number. Container price information refers to the transportation costs of different types of containers between specific routes and ports, which are usually divided according to the specifications and uses of the containers, such as standard containers, refrigerated containers, etc. Remarks information is an explanation attached to the basic information of shipping and the container price information, which may include general remarks applicable to all ports and special remarks for specific ports, such as transportation restrictions, specific requirements, etc., which help to improve and clarify the information.

[0065] Specifically, the first prompt instruction is received, and the instruction can be input into the large language model through the user interface or API call. The large language model can analyze the input data according to the instruction and extract the basic information of shipping. The first prompt instruction can include relevant concept descriptions, such as, "ETD date must exist, the format is generally `%m.%d`, and it is written together with the ship name, voyage or route code. If there is no ship name, voyage or route code, ETD will be directly marked near the date"; "Identify all the ship names, voyages, vessel / vol (not unique) in the information and their corresponding ETD dates (if the year information is missing, fill in the year closest to the current date) and route code route_code (usually 3 digits, may be empty, pay attention to distinguish the three-character port code). If there is no ship name, a record with an empty string for the ship name and voyage and today's ETD is constructed by default"; "Supplementary knowledge, commonly used domestic port cities (and abbreviations): Ningbo (NB), Shanghai (SH), Shekou (SK), Yantian (YT), Dachan Bay (DCB), Nansha (NS), Huangpu (HP), etc." Furthermore, the first prompt instruction may construct an empty value content, such as a default value of today / empty characters, etc.

[0066] After obtaining the basic information of shipping, the second prompt instruction is generated by parsing this information, which may involve constructing a query string or statement to explicitly ask for the price information of the container type. The instruction will then be input into the large language model again, and the instruction contains the required context to obtain the relevant container type price information. Exemplarily, the second prompt instruction can be "When the container type is not mentioned in the full text, "Price 1 / Price 2 / Price 3" respectively represent "20GP / 40GP / 40HQ"; if there is no "Price 3-40HQ", 40HQ defaults to the price of 40GP; if there is no "Price 2-40GP", 40GP defaults to the price of 40HQ"; "When `+` appears in the original information price, it means that this set of prices needs to be calculated based on the previous content"; "Additional cost information such as `ENS` / `PCS` does not need to be included in the container type price"

[0067] Similarly, based on the previous basic shipping information, the third prompt instruction is generated to obtain the notes related to each port. This process also calls the large language model to ensure that the instructions are accurate and have the necessary context so that the model can return valid notes.

[0068] In this embodiment, by inputting the first prompt instruction into the large language model and obtaining the basic shipping information, the core data can be automatically extracted, the speed and accuracy of information extraction can be improved, and the errors of manual input can be reduced, thereby improving the overall business efficiency. Automatically generating the second prompt instruction can ensure the full use of the basic shipping information, reduce the risk of information islands, and quickly and accurately combine this data with the basic information. Obtaining the remark information helps to provide additional context and ensure that all relevant factors are considered when arranging freight rates or transportation. By automatically generating instructions, not only is efficiency improved, but the accuracy and completeness of the remark information can also be ensured.

[0069] In one embodiment, the step of correcting the initial key information to obtain the corrected key information includes:

[0070] Obtain proofreading data on shipping information;

[0071] The initial key information is fuzzily matched with the shipping information proofreading data to obtain a first matching result, and based on the first matching result, misspelled information in the initial key information is modified, and completely wrong information in the initial key information is eliminated to obtain corrected key information.

[0072] Among them, shipping information proofreading data refers to a standard information set related to shipping, which is used to verify and correct the accuracy of the initial key information. These data usually include the port of departure, port of destination, ship name, voyage and its standard spelling, etc., as a basis for comparison. The first matching result is a preliminary fuzzy matching result set, which shows the degree of match between the initial key information and the shipping information proofreading data, and may contain correct matching and partial matching information. Corrected key information is an accurate key information set obtained after proofreading and correction, including information with corrected spelling errors and excluding incorrect data to ensure the validity and consistency of the data.

[0073] Specifically, first, the proofreading data of shipping information can be extracted from a predefined data source, such as a database, file, or API. After obtaining the proofreading data, the initial key information is compared with the proofreading data of shipping information using a fuzzy matching algorithm, such as Levenshtein distance, Jaccard similarity, etc. Each initial key information is compared with the corresponding item in the proofreading data to generate a first matching result to determine which information is similar, misspelled, or completely wrong. Based on the first matching result, the initial key information will be processed. For information that is successfully matched but has spelling errors, it will be replaced with the standard items in the proofreading data; for information that does not match at all, it will be eliminated. Finally, the corrected key information is generated to form a clean data set.

[0074] In this embodiment, fuzzy matching can effectively identify and correct spelling errors in the initial key information. By comparing the information with standard data, the accuracy of the data can be improved and the problems caused by manual input errors can be reduced. This process reduces the need for manual review and improves processing efficiency. The corrected key information after correction will significantly improve the accuracy and consistency of the data, ensuring that the data used in subsequent processing, analysis and decision-making is reliable.

[0075] In one embodiment, the steps of converting the non-standard port names in the corrected key information into standard port names by using a large language model and a pre-acquired port five-character code table to obtain the converted key information include:

[0076] The port name in the corrected key information is matched with the port name in the port five-character code table to obtain a second matching result, and based on the second matching result, the non-standard port name in the corrected key information is replaced with the standard port name in the port five-character code table to obtain the converted key information.

[0077] The second matching result is the result set obtained by matching the port names in the corrected key information with the port names in the port five-character code table, including the port names that have been matched successfully and the names that have not been matched, and identifying which names need to be replaced or corrected. The converted key information is the key information set in which the port names have been replaced with the standard names in the port five-character code table after standardization, ensuring the consistency and standardization of the information.

[0078] Specifically, first, the port name in the corrected key information is compared with the port name in the port five-character code table. This process uses fuzzy matching to identify the similarity between the two, traverses each port name in the corrected key information, and searches for the corresponding standard name in the port five-character code table. After the matching process is completed, a second matching result is generated to record the matching status of each port name. Among them, for the successfully matched names, the standard name is recorded, and for the unmatched names, they are marked as items that need further processing. After obtaining the second matching result, all non-standard port names in the corrected key information are replaced with the standard port names in the port five-character code table. By traversing the corrected key information, the successfully matched standard names are filled in the corresponding positions to form the converted key information.

[0079] In this embodiment, by matching the corrected key information with the port five-character code table, potential naming inconsistencies can be identified and corrected, helping to eliminate data confusion caused by spelling errors or the use of different names. Replacing the non-standard port names in the corrected key information with standard names not only improves the consistency and standardization of the data, but also simplifies subsequent data processing and analysis.

[0080] In one embodiment, according to the route correspondence table, the step of adjusting the port corresponding to each voyage name in the converted key information to obtain valid key information includes:

[0081] For each voyage in the converted key information, the voyage is matched with the route correspondence table to determine the various ports corresponding to the voyage, and the ports in the initial ports corresponding to the voyage that are inconsistent with the various ports are eliminated to obtain valid key information.

[0082] Among them, the ship name and voyage number refers to the ship name and voyage information of a specific shipping route, usually consisting of the ship name and voyage number, which is used to uniquely identify a specific shipping route.

[0083] Specifically, first traverse each voyage name in the converted key information and match it according to the route correspondence table. The matching process can be achieved through a search operation, and the corresponding port of passage is found in the route correspondence table according to the voyage name. For each successfully matched voyage name, extract its corresponding port of passage information. Next, compare the relationship between the initial port, that is, the port corresponding to the voyage name in the converted key information, and the port of passage. By traversing the initial port, check whether each port matches any port in the set of ports of passage. For unmatched ports, remove them to obtain valid key information.

[0084] In this embodiment, by matching the name of the voyage in the converted key information with the route correspondence table, the complete route of each voyage can be identified, ensuring that each name of the voyage can accurately reflect its actual transportation path. Extracting the port information can ensure that accurate shipping path data is used in subsequent processing. Eliminating the initial port that is inconsistent with the port of passage helps to ensure the accuracy and consistency of valid key information. After screening, a key information set that truly reflects the actual shipping situation can be generated to avoid decision-making errors caused by erroneous data.

[0085] In one embodiment, the step of using a large language model to verify the container type prices of ports with the same name in each entity relationship includes:

[0086] A large language model is used to verify the consistency of the container prices of ports with the same name in each entity relationship. If there are differences in the container prices of ports with the same name, the different container prices are screened and corrected according to the preset rules.

[0087] Among them, preset rules are rules used to guide and regulate how to deal with price differences during data processing. These rules can be fixed logic or dynamically generated based on the results of data analysis.

[0088] Specifically, the large language model is first called to verify the consistency of the container prices of ports with the same name in each entity relationship. The price information of ports with the same name is extracted, and through text analysis and comparison, the container prices of ports with the same name are identified to be different. After the price difference is found, it will be processed according to the preset rules. For example, all prices with differences are recorded, and reasonable prices are filtered out according to rules such as priority, timestamp, source, etc. Prices that do not meet the preset rules will be corrected and replaced with the screened standard prices.

[0089] In this embodiment, by using a large language model to verify the consistency of container prices at ports with the same name, potential data errors or inconsistencies can be quickly identified, the quality and accuracy of the data can be improved, and decision-making errors or business problems caused by erroneous data can be avoided. After discovering price differences, using preset rules for screening and correction can ensure the consistency and standardization of data processing. By clearly defining processing rules, complex data situations can be handled more effectively, reducing manual intervention and improving processing efficiency.

[0090] In one embodiment, the step of generating a shipping rate table according to each entity relationship includes:

[0091] Generate a subtable corresponding to each entity relationship, and map each subtable through the Cartesian product method to obtain the ocean freight rate table.

[0092] Among them, subtables refer to smaller tables extracted from the main data set, which contain information about specific entity relationships. For example, `Port × Price Table`, `Port × Schedule Table`, `Port × Remarks Table`, `Price × Schedule Table`, `Port × Five-Character Code Table`. Cartesian product is a mathematical operation used to combine elements from two or more sets. For each element in each set, the Cartesian product generates a new combination. For example, if there are two sets A and B, the Cartesian product A×B will generate all possible combinations of elements.

[0093] Specifically, relevant information is extracted from entity relationships and a subtable is generated for each entity relationship. This can be achieved through data query and filtering functions, extracting data from the main data set by conditions and organizing it into small tables. For example, a subtable containing all ports and their corresponding prices is generated from the port and price data. After all subtables are generated, they are mapped using the Cartesian product method. All rows in each subtable are traversed, combined with related data from other subtables to form all possible combinations. This process can be achieved through nested loops or using join operations in the database.

[0094] In this embodiment, by generating subtables, complex raw data can be broken down into smaller, more manageable parts. Using Cartesian products to map the subtables can produce a comprehensive and detailed ocean freight rate table, which ensures that all relevant information is fully integrated.

[0095] The following is a description of the shipping freight text recognition device based on a large language model provided in an embodiment of the present application. The shipping freight text recognition device based on a large language model described below and the shipping freight text recognition method based on a large language model described above can be referred to each other. Figure 2 As shown, the present application provides a device for recognizing shipping freight text based on a large language model, the device comprising:

[0096] The shipping freight rate text acquisition module 201 is used to acquire the shipping freight rate text of international logistics, and use the large language model to extract the initial key information and remark information from the shipping freight rate text. The initial key information includes the departure port, the destination port, the ship name, the voyage number and the container type price;

[0097] The port name standardization module 202 is used to modify the initial key information to obtain the modified key information, and convert the non-standard port name in the modified key information into a standard port name by using the pre-acquired port five-character code table through the large language model to obtain the converted key information;

[0098] The target key information determination module 203 is used to obtain a route correspondence table for indicating the correspondence between the ship name and the port, and adjust the port corresponding to each ship name and the port in the converted key information according to the route correspondence table to obtain valid key information, and supplement the valid key information with the remark information to obtain the target key information;

[0099] The ocean freight rate table generation module 204 is used to generate multiple entity correspondences based on the target key information using a large language model, and after verifying the container prices of the same-name ports in each entity relationship using the large language model, generate an ocean freight rate table according to each entity relationship.

[0100] In one embodiment, the shipping freight rate text acquisition module 201 includes:

[0101] A basic shipping information acquisition unit is used to acquire a first prompt instruction and input the first prompt instruction into a large language model to obtain basic shipping information, where the basic shipping information includes a departure port, a destination port, and a ship name and voyage number;

[0102] An initial key information determination unit is used to determine a second prompt instruction according to the basic shipping information, and input the second prompt instruction into the large language model to obtain container type price information, so as to use the basic shipping information and the container type price information as initial key information, and the container type price information includes the container type price of each port;

[0103] The remark information acquisition unit is used to determine the third prompt instruction according to the basic shipping information, and input the third prompt instruction into the large language model to obtain the remark information, which includes the general remark of each port and the special remark of a specific port.

[0104] In one embodiment, the port name standardization module 202 includes:

[0105] A shipping information proofreading data acquisition unit, used to acquire shipping information proofreading data;

[0106] The corrected key information determination unit is used to perform fuzzy matching on the initial key information and the shipping information proofreading data to obtain a first matching result, and based on the first matching result, to modify misspelled information in the initial key information and eliminate completely wrong information in the initial key information to obtain the corrected key information.

[0107] In one embodiment, the port name standardization module 202 includes:

[0108] The port name standardization unit is used to match the port name in the corrected key information with the port name in the port five-character code table to obtain a second matching result, and based on the second matching result, use the standard port name in the port five-character code table to replace the non-standard port name in the corrected key information to obtain the converted key information.

[0109] In one embodiment, the target key information determination module 203 includes:

[0110] The effective key information determination unit is used to match each voyage name in the converted key information with the route correspondence table, determine the various ports corresponding to the voyage name, and eliminate the ports in the initial ports corresponding to the voyage name that are inconsistent with the various ports, so as to obtain effective key information.

[0111] In one embodiment, the ocean freight rate table generating module 204 includes:

[0112] The container price verification unit is used to use a large language model to verify the consistency of the container prices of ports with the same name in each entity relationship. If there are differences in the container prices of ports with the same name, the different container prices will be screened and corrected according to preset rules.

[0113] In one embodiment, the ocean freight rate table generating module 204 includes:

[0114] The ocean freight rate table generating unit is used to generate a subtable corresponding to each entity relationship, and map each subtable by a Cartesian product method to obtain an ocean freight rate table.

[0115] In one embodiment, the present application further provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for recognizing shipping freight rates text based on a large language model as described in any of the above embodiments.

[0116] In one embodiment, the present application further provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for recognizing shipping freight rates text based on a large language model as described in any of the above embodiments.

[0117] Indicatively, Figure 3 As shown, Figure 3 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 300 may be provided as a server. Figure 3 The computer device 300 includes a processing component 302, which further includes one or more processors, and a memory resource represented by a memory 301, for storing instructions executable by the processing component 302, such as an application. The application stored in the memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 302 is configured to execute instructions to perform the method for ocean freight rate text recognition based on a large language model of any of the above embodiments.

[0118] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0119] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0120] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, "one", "one", "said", "the" and "it" may also include plural forms, unless the context clearly indicates another way. A plurality refers to at least two cases, such as 2, 3, 5 or 8, etc. "And / or" includes any and all combinations of the relevant listed items.

[0121] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0122] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for recognizing shipping freight rates text based on a large language model, characterized in that: The method comprises: Obtain the text of international logistics ocean freight rates, and use a large language model to extract initial key information and remark information from the ocean freight rate text, where the initial key information includes the port of departure, port of destination, ship name, voyage number, and container type price; The initial key information is modified to obtain modified key information, and the non-standard port name in the modified key information is converted into a standard port name by using the large language model and the pre-acquired port five-character code table to obtain converted key information; Obtaining a route correspondence table for indicating the correspondence between a ship name and a voyage and a port, adjusting the port corresponding to each voyage name in the converted key information according to the route correspondence table to obtain valid key information, and supplementing the valid key information with the remark information to obtain target key information; According to the target key information, the large language model is used to generate multiple entity correspondences, and after the container prices of the ports with the same name in each of the entity relationships are verified by the large language model, a shipping freight rate table is generated according to each of the entity relationships.

2. The method for recognizing shipping freight rates based on a large language model according to claim 1, characterized in that: The step of extracting initial key information and remark information from the shipping freight rate text using a large language model includes: Obtaining a first prompt instruction, and inputting the first prompt instruction into the large language model to obtain basic shipping information, wherein the basic shipping information includes a departure port, a destination port, and a ship name and voyage number; Determine a second prompt instruction according to the basic shipping information, and input the second prompt instruction into the large language model to obtain container type price information, so as to use the basic shipping information and the container type price information as the initial key information, and the container type price information includes the container type price of each port; A third prompt instruction is determined according to the basic shipping information, and the third prompt instruction is input into the large language model to obtain the remark information, wherein the remark information includes general remark of each port and special remark of a specific port.

3. The method for recognizing shipping freight rates based on a large language model according to claim 1, characterized in that: The step of correcting the initial key information to obtain the corrected key information includes: Obtain proofreading data on shipping information; The initial key information is fuzzily matched with the shipping information proofreading data to obtain a first matching result, and based on the first matching result, misspelled information in the initial key information is modified, and completely wrong information in the initial key information is eliminated to obtain the corrected key information.

4. The method for recognizing shipping freight rates based on a large language model according to claim 1, characterized in that: The step of converting the non-standard port name in the corrected key information into a standard port name by using the large language model and the pre-acquired port five-character code table to obtain the converted key information comprises: The port name in the corrected key information is matched with the port name in the port five-character code table to obtain a second matching result, and based on the second matching result, the non-standard port name in the corrected key information is replaced with the standard port name in the port five-character code table to obtain the converted key information.

5. The method for recognizing shipping freight rates based on a large language model according to claim 1, characterized in that: The step of adjusting the port corresponding to each voyage name in the converted key information according to the route correspondence table to obtain valid key information includes: For each voyage in the converted key information, the voyage is matched with the route correspondence table to determine the various ports along the way corresponding to the voyage, and the ports in the initial ports corresponding to the voyage that are inconsistent with the various ports along the way are eliminated to obtain the valid key information.

6. The method for recognizing shipping freight rates based on a large language model according to claim 1, characterized in that: The step of using the large language model to verify the container type prices of ports with the same name in each of the entity relationships includes: The large language model is used to verify the consistency of the container prices of the ports with the same name in each of the entity relationships. If there are differences in the container prices of the ports with the same name, the container prices with differences are screened and corrected according to preset rules.

7. The method for recognizing shipping freight rates based on a large language model according to any one of claims 1 to 6, characterized in that: The step of generating a shipping freight rate table according to each of the entity relationships comprises: Generate a subtable corresponding to each entity relationship, and map each subtable by a Cartesian product method to obtain the ocean freight rate table.

8. A device for recognizing shipping freight rates based on a large language model, characterized in that: The device comprises: The shipping freight rate text acquisition module is used to obtain the shipping freight rate text of international logistics, and use the large language model to extract the initial key information and remark information from the shipping freight rate text. The initial key information includes the departure port, destination port, ship name, voyage number and container type price; A port name standardization module is used to modify the initial key information to obtain modified key information, and convert the non-standard port name in the modified key information into a standard port name by using the pre-acquired port five-character code table through the large language model to obtain the converted key information; A target key information determination module is used to obtain a route correspondence table for indicating the correspondence between a ship name and a voyage and a port, and adjust the port corresponding to each voyage in the converted key information according to the route correspondence table to obtain valid key information, and use the remark information to supplement the valid key information to obtain target key information; The module for generating a shipping freight rate table is used to generate a plurality of entity correspondences according to the target key information by using the large language model, and to generate a shipping freight rate table according to each entity relationship after verifying the container type prices of the ports with the same name in each entity relationship by using the large language model.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for recognizing shipping freight rates text based on a large language model as described in any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the method for recognizing shipping freight rates text based on a large language model as described in any one of claims 1 to 7 are performed.

Images