Trip leg information extraction and verification method and system based on programmable large model

Abstract

The present application relates to the technical field of data processing, and in particular to a trip leg information extraction and verification method and system based on a programmable large model, which comprises the following steps: extracting a set of leg text blocks of each leg from a plurality of travel documents to be processed; determining the field uncertainty of each field of each leg according to the occurrence of each field in the set of leg text blocks of each leg; assigning a corresponding programmable large model to each field of each leg, thereby improving the field extraction efficiency and accuracy; then determining the spatio-temporal deviation of each leg; determining the abnormal leg from the spatio-temporal deviation of each leg; and finally adjusting the programmable large model of the abnormal leg to further improve the accuracy of field information extraction of the abnormal leg.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a method and system for extracting and verifying travel segment information based on a programmable large model. Background Technology

[0002] In travel information extraction, it is necessary to extract flight segment information from various document formats submitted by staff (such as PDF itineraries, email screenshots, e-tickets, etc.), including key fields such as departure point, destination, flight number, departure time, arrival time, and cabin class. Programmable large models refer to the ability to organize multiple complementary large models into a collaborative system through scheduling. This system can dynamically select, combine, and call the most suitable model to execute tasks based on specific task characteristics, thereby improving efficiency and accuracy.

[0003] Existing methods primarily employ a single large model for text recognition and information extraction of flight segment information. However, using a single large model cannot effectively handle information conflicts between multi-source heterogeneous documents because it lacks an assessment of field credibility. When flight numbers or times conflict in documents from different sources, it is impossible to determine which information is more credible. Furthermore, when using a single large model to process all fields, the lack of differentiation leads to low extraction accuracy. It also lacks a deep understanding of the logical relationships between fields and domain knowledge, making it difficult to effectively identify and verify contradictions in the information. For example, time fields require standardized formats, location fields require geocoding, and flight numbers need to conform to IATA encoding rules. Summary of the Invention

[0004] To address the low accuracy problem of existing methods for extracting travel segment information, this invention aims to provide a method and system for extracting and verifying travel segment information based on a programmable large model. The specific technical solution adopted is as follows: In a first aspect of the present invention, a method for extracting and verifying travel segment information based on a programmable large model is provided, comprising: Extract the set of text blocks for each flight segment from multiple travel documents to be processed; Based on the occurrence of each field in the set of text blocks of each flight segment, the field uncertainty of each field in each flight segment is determined; Based on the uncertainty of the fields, a corresponding programmable large model is assigned to each field of each segment to indicate that each field of each segment is arranged according to the corresponding programmable large model and output the field target candidate value of each field of each segment. Based on the candidate values ​​of the field targets for each segment, the spatiotemporal deviation of each segment is determined; Abnormal flight segments are determined by the spatiotemporal deviation of each flight segment; Adjust the large programmable model of the abnormal flight segment to indicate the reprogramming extraction of the flight segment information of the abnormal flight segment.

[0005] In an exemplary embodiment, extracting a set of flight segment text blocks from multiple travel documents to be processed includes: The text of each travel document to be processed is extracted to obtain the original text sequences. The original text sequence is segmented into text blocks to obtain flight segment text blocks; Group the text blocks of the same flight segment together to obtain the set of text blocks of each flight segment.

[0006] In an exemplary embodiment, determining the field uncertainty of each field for each flight segment includes: Determine the text consistency of each field in each flight segment; the text consistency represents the situation where the same field value appears most frequently in the corresponding flight segment text block set; Determine the domain rule compliance degree of each field in each flight segment; the domain rule compliance degree represents the situation in which each field of each flight segment satisfies the corresponding field rule in the corresponding flight segment text block set; The field uncertainty is obtained based on the text consistency and domain rule compliance; the field uncertainty is inversely correlated with both the text consistency and domain rule compliance.

[0007] In an exemplary embodiment, the process of obtaining text consistency includes: Get the maximum number of occurrences of any field in any given flight segment within the corresponding set of flight segment text blocks; The ratio of the maximum occurrence count to the number of segment text blocks in the corresponding segment text block set is used as the text consistency of any field in any segment. The process of obtaining the domain rule compliance includes: Get the number of times a field in any given flight segment satisfies the corresponding field rule in the corresponding flight segment text block set; The ratio of the number of elements satisfying the corresponding field rule to the number of elements in the corresponding segment text block set is used as the domain rule compliance degree of any field of any segment.

[0008] In one exemplary embodiment, the programmable large model is divided into a lightweight extraction model, a standard understanding model, and a deep inference model; The assignment of corresponding programmable large models to each field of each flight segment based on field uncertainty includes: Compare the field uncertainty of any field in any flight segment with the magnitudes of a first threshold and a second threshold; the first threshold is greater than the second threshold. When the uncertainty of the field is less than the second threshold, a lightweight extraction model is assigned to any field of any flight segment; When the uncertainty of the field is less than the first threshold and greater than or equal to the second threshold, a standard understanding model is assigned to any field of any flight segment; When the uncertainty of the field is greater than or equal to the first threshold, a deep inference model is assigned to any field of any flight segment.

[0009] In an exemplary embodiment, when arranging numeric fields according to the deep inference model, the standard understanding model is also called simultaneously for arrangement, outputting two candidate values ​​for the fields; the two output candidate values ​​for the fields are weighted and fused to obtain the corresponding target candidate values ​​for the fields.

[0010] In an exemplary embodiment, determining the spatiotemporal deviation of each flight segment based on the candidate values ​​of the field targets for each field of each flight segment includes: Determine the time difference between the theoretical flight time and the actual flight time for any given flight segment; Determine the spatial continuity indicator variable between any flight segment and its adjacent next flight segment; wherein, if the distance between the arrival point of any flight segment and the departure point of its adjacent next flight segment is greater than a preset distance, the spatial continuity indicator variable is a first preset value, otherwise it is a second preset value, and the first preset value is greater than the second preset value. Determine the time continuity indicator variable between any flight segment and its adjacent next flight segment; wherein, if the transfer time between any flight segment and its adjacent next flight segment is less than a preset time, the time continuity indicator variable is the first preset value, otherwise it is the second preset value; By integrating the time difference, spatial continuity indicator variable, and time continuity indicator variable, the spatiotemporal deviation of any flight segment is obtained; the spatiotemporal deviation is positively correlated with the time difference, spatial continuity indicator variable, and time continuity indicator variable.

[0011] In an exemplary embodiment, determining abnormal flight segments based on the spatiotemporal deviation of each flight segment includes: Flight segments with a spatiotemporal deviation greater than the spatiotemporal deviation anomaly threshold are identified as the abnormal flight segments.

[0012] In one exemplary embodiment, adjusting the programmable large model of the abnormal flight segment includes: The programmable large model of each field of the abnormal flight segment is adjusted to a deep inference model.

[0013] In a second aspect of the present invention, a travel segment information extraction and verification system based on a programmable large model is provided, comprising: a memory and a processor; the memory is connected to the processor; the memory is used to store program instructions; the processor is used to implement the above-mentioned travel segment information extraction and verification method based on a programmable large model when the program instructions are executed.

[0014] This invention has the following beneficial effects: By constructing uncertainty indicators for each field of each flight segment, this invention quantifies the uncertainty of each field in each flight segment, thereby providing an objective basis for the allocation and arrangement of the large-scale programmable models for each field in each flight segment. This avoids the problems of low extraction efficiency and low extraction accuracy caused by using the same processing strategy for all fields of all flight segments in traditional methods, and improves the targeting of information extraction. By allocating appropriate large-scale programmable models for each field of each flight segment, it saves computational resources while ensuring extraction accuracy and improves the robustness of extraction results. By determining the spatiotemporal deviation of each flight segment, the large-scale programmable models for abnormal flight segments are adjusted, making the large-scale programmable models more compatible with the flight segments, further improving the accuracy of field information extraction for abnormal flight segments. Attached Figure Description

[0015] Figure 1 This is a flowchart of a method for extracting and verifying travel segment information based on a programmable large model, provided in one embodiment of the present invention; Figure 2 This is a flowchart illustrating the implementation of step S1 provided in one embodiment of the present invention; Figure 3 This is a flowchart illustrating the process of obtaining field uncertainty according to an embodiment of the present invention; Figure 4 This is a flowchart of the process for obtaining spatiotemporal deviation provided in one embodiment of the present invention. Detailed Implementation

[0016] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. All data and information collected in this application have been obtained with full consent.

[0018] like Figure 1As shown, this embodiment provides a method for extracting and verifying travel segment information based on a programmable large model, including the following steps: Step S1: Extract the set of text blocks for each flight segment from the multiple travel documents to be processed; Step S2: Determine the field uncertainty of each field in each segment based on the occurrence of each field in the segment text block set of each segment; Step S3: Assign corresponding programmable large models to each field of each flight segment based on field uncertainty, so as to indicate that each field of each flight segment is arranged according to the corresponding programmable large models and output the field target candidate values ​​of each field of each flight segment; Step S4: Determine the spatiotemporal deviation of each flight segment based on the candidate values ​​of the field targets for each field. Step S5: Determine abnormal flight segments based on the spatiotemporal deviation of each flight segment; Step S6: Adjust the large-scale programmable model of the abnormal segments to indicate the segment information of the extracted abnormal segments for reprogramming.

[0019] The following detailed explanation of each step, in conjunction with the accompanying drawings, is provided.

[0020] Step S1: Extract the set of flight segment text blocks for each flight segment from the multiple travel documents to be processed.

[0021] Retrieve multiple travel documents to be processed, setting the number of travel documents to N. Extract the set of flight segment text blocks for each flight segment from the multiple travel documents to be processed. It should be understood that since a single trip may contain multiple flight segments, a single travel document may contain information for multiple flight segments.

[0022] In one exemplary embodiment, such as Figure 2 As shown, the following is a specific implementation process of step S1: Step S11: Extract text from each travel document to be processed to obtain each original text sequence.

[0023] For any travel document, text extraction is performed. In an exemplary embodiment, a document parsing model (e.g., MonkeyOCR v1.5) is used to extract the text from the travel document, converting the document content into text format, and extracting the original text sequence of the travel document.

[0024] Step S12: Perform text block segmentation on the original text sequence to obtain flight segment text blocks.

[0025] For each flight segment, the fields to be extracted should be flexibly selected based on the actual situation. Usually, more critical fields are selected, such as, but not limited to: departure point, destination, flight number, departure time and arrival time, etc.

[0026] It should be understood that each flight segment may appear repeatedly in multiple travel documents, but some information may be lost during the extraction process. Let the number of times the k-th flight segment appears in all travel documents be denoted as . .

[0027] The original text sequence is segmented into text blocks to obtain flight segment text blocks. Each flight segment text block contains flight segment information. Since a single trip may contain multiple flight segments, for any given original text sequence, the flight number information is determined, for example, by using regular expressions to match strings conforming to the IATA flight number format. Then, following the text order in the original text sequence, each string containing a flight number is searched for. The content from one string containing a flight number to the beginning of the next string containing a flight number is then considered as one flight segment text block, thus obtaining several flight segment text blocks.

[0028] Therefore, for any given flight segment text block, there are fields such as departure location, destination, flight number, departure time, and arrival time. In an exemplary embodiment, a named entity recognition model is used to recognize the strings of the fields such as departure location, destination, flight number, departure time, and arrival time in each flight segment text block.

[0029] Step S13: Group the segment text blocks of the same segment into a group to obtain the segment text block set of each segment.

[0030] Step S12 yields several segment text blocks in each original text sequence. Since the same segment may appear repeatedly in multiple original text sequences, segment text blocks of the same segment are grouped together. That is, the field strings of several segment text blocks with the same flight number are grouped together to form a set of segment text blocks for each segment. The number of segment text blocks in the set is the number of times the same segment appears in multiple original text sequences.

[0031] Step S2: Determine the field uncertainty of each field in each segment based on the occurrence of each field in the segment text block set of each segment.

[0032] For the k-th flight segment, since it appears the most times in all travel documents... Then the set of segment text blocks for the k-th segment contains data from... Each travel document contains a segment text block. Each field in the segment text block may be identified as multiple different candidate values, which may include conflicts, errors, or redundancies. The multiple different candidate values ​​corresponding to each field are determined as the candidate value set for each field.

[0033] For ease of explanation, taking the i-th field of the k-th segment as an example, the field uncertainty of the i-th field of the k-th segment is determined based on the occurrence of the i-th field in the segment text block set of the k-th segment. Field uncertainty characterizes the error situation during field recognition. In an exemplary embodiment, such as... Figure 3 As shown, the following is a specific process for obtaining field uncertainty: Step S21: Determine the text consistency of each field in each flight segment.

[0034] The text consistency of the i-th field in the k-th segment is determined. Text consistency represents the situation where the i-th field of the k-th segment has the most frequent occurrences of the same field value in the segment text block set of the k-th segment. In an exemplary embodiment, since the i-th field of the k-th segment may be identified as multiple different candidate values ​​(i.e., multiple different field values) in multiple segment text blocks within its segment text block set, the occurrence counts of the same field value among the multiple field values ​​corresponding to the i-th field of the k-th segment are counted to obtain the occurrence count of each field value. The maximum occurrence count is then obtained and set as the maximum occurrence count of the same field value corresponding to the i-th field of the k-th segment. . This indicates the maximum number of times the field value of the i-th field in the k-th segment is the same across all segment text blocks. (Maximum occurrence count) The more [number] segments, the higher the text consistency of the i-th field in the k-th segment. This is because the number of segment text blocks in the i-th field of the k-th segment is [number]. ,but This represents the maximum theoretical number of occurrences of a field value when all values ​​in the i-th field of the k-th flight segment are identical. Therefore, we calculate the maximum number of occurrences corresponding to identical values ​​in the i-th field of the k-th flight segment. and The ratio is used to maximize the occurrence of the same field value in the i-th field of the k-th segment. The normalization will and The ratio is used as the text consistency of the i-th field in the k-th segment. The higher the text consistency, the more consistent the field value of the i-th field in the k-th segment is across different travel document sources, and the lower the field uncertainty of the i-th field in the k-th segment. Field uncertainty is inversely correlated with text consistency.

[0035] Step S22: Determine the domain rule compliance of each field in each flight segment.

[0036] Determine the domain rule compliance degree of the i-th field of the k-th segment. The domain rule compliance degree represents the degree to which the i-th field of the k-th segment satisfies the field rule of the i-th field in the segment text block set of the k-th segment. The more the corresponding field rule is satisfied, the higher the domain rule compliance degree.

[0037] In an exemplary embodiment, corresponding domain rule detection methods are defined for different types of fields. For example, for the flight number field, regular expression matching is used to check whether it conforms to the standard format of IATA two-letter airline code plus numbers; for the time field, it is checked whether it is a valid date and time format and within a reasonable range; for the location field, it is checked whether it can match the standard three-letter airport code. It should be understood that the definition of the domain rule detection methods for the above-mentioned types of fields are existing rules and will not be elaborated further. It should be noted that the corresponding field rules are also the domain rule detection methods.

[0038] For the i-th field of the k-th segment, determine whether the identified field value in each segment text block of its segment text block set satisfies the corresponding domain rule detection method, and determine the number of fields that satisfy the corresponding domain rule detection method, defined as... It should be understood that if all the field values ​​identified in the i-th field of the k-th flight segment satisfy the corresponding domain rule detection method, then the number of fields satisfying the domain rule detection method is... ,but This represents the maximum theoretical number of cases where the field value of the i-th field in the k-th flight segment satisfies the domain rule detection method. Then, calculate... and The ratio is used to determine the i-th field corresponding to the k-th flight segment. Normalization, and The ratio is used as the domain rule compliance degree of the i-th field of the k-th segment. The larger the domain rule compliance degree value, the more the field value of the i-th field of the k-th segment conforms to the domain specification, and the lower the field uncertainty of the i-th field of the k-th segment. Field uncertainty is inversely correlated with domain rule compliance degree.

[0039] Step S23: Based on text consistency and domain rule compliance, obtain field uncertainty.

[0040] By combining the text consistency and domain rule compliance of the i-th field in the k-th flight segment, the field uncertainty of the i-th field in the k-th flight segment is obtained. Based on the above logical analysis, the following is a method for calculating the field uncertainty of the i-th field in the k-th flight segment: ; in, This indicates the uncertainty of the i-th field in the k-th flight segment; This indicates the text consistency of the i-th field in the k-th flight segment; This represents the domain rule compliance degree of the i-th field in the k-th flight segment. By averaging, text consistency and domain rule compliance are combined to obtain the field uncertainty.

[0041] Step S3: Assign corresponding programmable large models to each field of each segment based on field uncertainty, so as to indicate that each field of each segment is arranged according to the corresponding programmable large models and output the field target candidate values ​​of each field of each segment.

[0042] First, we construct a programmable large model pool, containing three types of programmable large models with functional specializations (i.e., large models optimized for specific tasks or domains, unlike general-purpose models): lightweight extraction models. Standard Understanding Model and deep reasoning models In one exemplary embodiment, a lightweight extraction model This is a BERT-based named entity recognition model specifically designed for handling fields with standardized formats and low uncertainty, such as standardized flight numbers and airport codes. Its advantages include fast inference speed and low resource consumption; a standard understanding model. This could be a Transformer model, specifically a medium-sized Transformer model, capable of understanding natural language descriptions and extracting information, suitable for fields with intermediate levels of uncertainty; or a deep inference model. It can be used for large-scale language models, with powerful logical reasoning and knowledge association capabilities, and can handle fields with high uncertainty, such as those with ambiguous expressions, conflicts, or those requiring cross-field reasoning.

[0043] Based on the uncertainties of each field in each flight segment, a suitable programmable large model is assigned to each field in each flight segment. Based on the analysis above, this embodiment sets two thresholds: a first threshold and a second threshold, with the first threshold being greater than the second threshold. Both the first and second thresholds have a numerical range of 0-1, and their specific values ​​are set according to actual judgment needs; for example, the first threshold could be 0.5 and the second threshold 0.3. It should be noted that both the first and second thresholds are determined through analysis of historical data.

[0044] Taking the i-th field of the k-th segment as an example, compare the field uncertainty of the i-th field of the k-th segment with the size of the first threshold and the second threshold.

[0045] When the uncertainty of the i-th field in the k-th segment is less than the second threshold, it indicates that the uncertainty of the i-th field in the k-th segment is low, and a lightweight extraction model is assigned to the i-th field in the k-th segment. Arrange lightweight extraction models Process it.

[0046] When the uncertainty of the i-th field in the k-th segment is less than the first threshold and greater than or equal to the second threshold, a standard understanding model is assigned to the i-th field of the k-th segment. Layout Standard Understanding Model Process it.

[0047] When the uncertainty of the i-th field in the k-th segment is greater than or equal to the first threshold, a deep inference model is assigned to the i-th field of the k-th segment. Crafting deep reasoning models Processing is then performed. Furthermore, when the uncertainty of the i-th field in the k-th segment is greater than or equal to the first threshold, in addition to orchestrating the deep inference model... In addition to processing, the standard understanding model can also be invoked simultaneously. The program is arranged to achieve parallel extraction of the two models, and then the outputs of the two models are fused.

[0048] Therefore, for each field of each flight segment with medium or low uncertainty, input the set of flight segment text blocks corresponding to each flight segment, or input the set of candidate values ​​corresponding to each field of each flight segment, to obtain the target candidate values ​​of each field of each flight segment.

[0049] For each field of each flight segment with high field uncertainty, the standard interpretation model is used. and deep reasoning models Extracting values ​​from each field separately yields two candidate values. It should be understood that this is done while utilizing a standard interpretation model. and deep reasoning models Parallel field extraction is suitable for numeric fields. For numeric fields with high uncertainty, a standard interpretation model can also be used. and deep reasoning models If two candidate values ​​for a field are obtained, and these two candidate values ​​are both numerical, then a weighted summation is performed on these two values, and the result is used as the target candidate value for that field. In an exemplary embodiment, taking the i-th field of the k-th flight segment as an example, if the i-th field of the k-th flight segment is a numerical field with high uncertainty, then the formula for calculating the target candidate value of the i-th field of the k-th flight segment is as follows: ; in, This represents the candidate value of the target field in the i-th field of the k-th flight segment. Indicates the use of deep reasoning models The extracted candidate values ​​for the i-th field of the k-th flight segment. This indicates the use of the standard understanding model. The extracted candidate values ​​for the i-th field of the k-th flight segment. express The weighting coefficients, express The weighting coefficients, It should be understood that The higher the proportion, The higher the value, the better. and The value is set according to actual needs. In an exemplary embodiment, the uncertainty of the i-th field of the k-th segment can be... As .

[0050] In summary, the corresponding programmable large model is used for programming, and the target candidate values ​​of each field of each flight segment are output.

[0051] Because named entity recognition models can only provide isolated, raw, and potentially conflicting strings, they cannot understand the logical relationships between fields, resolve conflicts between multi-source data, or handle non-standard expressions. This step, through the deep semantic understanding and reasoning capabilities of a large model, can verify different candidate values ​​for each field and ultimately output the target candidate value for the field.

[0052] Step S4: Determine the spatiotemporal deviation of each flight segment based on the candidate values ​​of the field targets for each field of each flight segment.

[0053] After obtaining the target candidate values ​​for each field of each flight segment, although field extraction has been completed, logical errors may still exist in the extraction results, such as mismatches between flight time and flight distance, insufficient transfer time between adjacent flight segments, and other violations of physical laws. These errors cannot be detected solely by textual information and require cross-validation between the extracted flight segment information and real-world spatiotemporal physical laws. Therefore, this step needs to determine the spatiotemporal deviation of each flight segment based on the target candidate values ​​for each field, automatically identify unreasonable target candidate values, and trigger a model re-arrangement mechanism for correction, thereby completing closed-loop verification. In an exemplary embodiment, such as... Figure 4 As shown, the following is a specific process for obtaining the spatiotemporal deviation: Step S41: Determine the time difference between the theoretical flight time and the actual flight time for any flight segment.

[0054] It should be understood that the various parameters mentioned below are various fields, and the values ​​of each parameter are the target candidate values ​​of each field extracted through step S3.

[0055] The analysis is performed using the k-th flight segment as any segment. The theoretical flight time of the k-th segment is obtained. In an exemplary embodiment, the three-letter airport codes of the origin and destination of the k-th segment are extracted, the latitude and longitude coordinates are obtained by querying a standard airport geographic database, and the spherical distance of the k-th segment is calculated using the Haversine formula. This refers to the flight distance of the k-th segment. The equivalent flight speed is determined based on the typical cruising speed range of civil airliners (850-900 km / h). For example, the median of the typical cruise speed range can be used as the equivalent flight speed. Based on equivalent flight speed Calculate the theoretical flight time for the k-th segment. , If the spherical distance is less than or equal to the minimum distance threshold, the k-th segment is directly identified as an abnormal segment; the minimum distance threshold is set to 1, and in other embodiments, the implementer may adjust this value according to the actual situation.

[0056] The actual flight time of the k-th segment is determined based on the obtained departure and arrival times. Based on the theoretical flight time of the k-th segment... and actual flight time The theoretical flight time for the k-th segment is calculated. and actual flight time In an exemplary embodiment, the time difference is calculated as follows: ; in, This represents the theoretical flight time for the k-th segment. and actual flight time The time difference. The spatiotemporal consistency deviation of the k-th flight segment is represented by a dimensionless relative time deviation. It can verify the correspondence between flight time and flight distance. If the time field or location field of the extracted k-th flight segment is incorrect, it will lead to a mismatch between the actual flight time and the theoretical flight time. The larger the value, the higher the probability of field errors in the k-th flight segment, and the higher the spatiotemporal deviation of the k-th flight segment. The spatiotemporal deviation of the k-th flight segment is related to... Positive correlation.

[0057] It should be understood that, under normal circumstances, the difference between the theoretical and actual flight time of a flight segment is small. If the difference is too large, the flight segment may have a serious field error or other serious anomalies. Therefore, a time difference threshold is preset, with a value ranging from 0 to 1, and a value close to 1 (e.g., 0.8) can be set. When the time difference of the k-th flight segment... If the time difference threshold is greater than or equal to the threshold, the kth segment can be directly identified as an abnormal segment, and the following calculation of the spatiotemporal deviation will not be performed.

[0058] Step S42: Determine the spatial continuity indicator variable between any segment and its adjacent next segment.

[0059] The next flight segment adjacent to the k-th flight segment is designated as the (k+1)-th flight segment. A spatial continuity indicator variable is determined between the k-th and (k+1)-th flight segments. This spatial continuity indicator variable characterizes the spatial continuity between the k-th and (k+1)-th flight segments. In an exemplary embodiment, the arrival point of the k-th flight segment and the departure point of the (k+1)-th flight segment are extracted, and the distance between them is determined. This embodiment sets a preset distance to determine whether the distance between the arrival point of the k-th flight segment and the departure point of the (k+1)-th flight segment is too far. This preset distance can be obtained empirically; in this embodiment, 50 kilometers is used as an example. Compare the distance between the destination of the k-th segment and the departure point of the (k+1)-th segment with a preset distance. If the distance is greater than the preset distance, it indicates that the distance between the destination of the k-th segment and the departure point of the (k+1)-th segment is relatively large, meaning the k-th and (k+1)-th segments are spatially discontinuous. The spatial continuity indicator variable for the k-th and (k+1)-th segments is high, and the spatial continuity indicator variable is set to the first preset value. If the distance is less than or equal to the preset distance, it indicates that the distance between the destination of the k-th segment and the departure point of the (k+1)-th segment is relatively small, meaning the k-th and (k+1)-th segments are spatially continuous. The spatial continuity indicator variable for the k-th and (k+1)-th segments is low, and the spatial continuity indicator variable is set to the second preset value. The first preset value is greater than the second preset value; the value by which the first preset value is greater than the second preset value is set according to actual needs, in order to... Being on the same order of magnitude facilitates subsequent quantification. In an exemplary embodiment, the first preset value is set to 1, and the second preset value is set to 0. When the value of the spatial continuity indicator variable corresponding to the k-th flight segment is high, it indicates that the probability of field errors in the k-th flight segment is higher, the spatiotemporal deviation of the k-th flight segment is higher, and the spatiotemporal deviation of the k-th flight segment is positively correlated with the spatial continuity indicator variable.

[0060] The spatial continuity indicator variable can also be represented as a penalty term for adjacent flight segments in terms of spatial continuity. From the perspective of travel logic, adjacent flight segments must satisfy the following conditions: the distance between the destination of the preceding segment and the departure point of the following segment is not significantly different, or the two locations are within the same city. When this constraint is not met, the spatial continuity indicator variable is set to 1.

[0061] Step S43: Determine the time continuity indicator variable between any segment and its adjacent next segment.

[0062] A time continuity indicator variable is determined between the k-th segment and the (k+1)-th segment. This time continuity indicator variable characterizes the temporal continuity between the k-th segment and the (k+1)-th segment. In an exemplary embodiment, the transfer time between the k-th segment and the (k+1)-th segment is determined. This embodiment sets a preset time, which is used to determine whether the transfer time between the k-th segment and the (k+1)-th segment is too long. This preset time can be obtained empirically; in this embodiment, one hour is used as an example. Compare the transfer time between the k-th segment and the (k+1)-th segment with the preset time. If the transfer time is less than the preset time, it indicates that the transfer time between the k-th segment and the (k+1)-th segment is relatively long, and the k-th segment and the (k+1)-th segment are not consecutive in time. The k-th segment and the (k+1)-th segment are a pair of discontinuous segments in time, and the time continuity indicator variable between the k-th segment and the (k+1)-th segment is high. Set the time continuity indicator variable to the first preset value (i.e., set it to 1). If the transfer time is greater than or equal to the preset time, it indicates that the k-th segment and the (k+1)-th segment are consecutive in time, and the time continuity indicator variable between the k-th segment and the (k+1)-th segment is low. Set the time continuity indicator variable to the second preset value (i.e., set it to 0). When the time continuity indicator variable corresponding to the k-th flight segment is high, it indicates that the field of the k-th flight segment is more likely to be incorrect, the spatiotemporal deviation of the k-th flight segment is higher, and the spatiotemporal deviation of the k-th flight segment is positively correlated with the time continuity indicator variable.

[0063] The time continuity indicator variable can also be represented as a penalty for the time continuity of adjacent flight segments. Based on the transfer rules, the transfer time between adjacent flight segments must meet the minimum physical requirements, including the total time for disembarking, passing through security, and arriving at the next gate. Therefore, the preset time is set to 1 hour. When this constraint is not met, the time continuity indicator variable is set to 1.

[0064] It should be understood that since the last flight segment does not have an adjacent subsequent flight segment, the spatial continuity indicator variable and the temporal continuity indicator variable are not calculated for the last flight segment, and only the time difference between the theoretical flight time and the actual flight time is obtained; or, no anomaly judgment is made for the last flight segment.

[0065] Step S44: Integrate the time difference, spatial continuity indicator variable and the time continuity indicator variable to obtain the spatiotemporal deviation of any flight segment.

[0066] The spatiotemporal deviation of the k-th flight segment is obtained by fusing the time difference, spatial continuity indicator variable, and temporal continuity indicator variable corresponding to the k-th flight segment. In an exemplary embodiment, the fusion method is as follows: the average value of the time difference, spatial continuity indicator variable, and temporal continuity indicator variable corresponding to the k-th flight segment is calculated, and the result is used as the spatiotemporal deviation of the k-th flight segment. Thus, the spatiotemporal deviation of each flight segment is obtained.

[0067] Step S5: Determine abnormal flight segments based on the spatiotemporal deviation of each flight segment.

[0068] The higher the spatiotemporal deviation of a flight segment, the more abnormal the segment. This embodiment presets a spatiotemporal deviation anomaly threshold, which is used to compare with the spatiotemporal deviation of each flight segment to determine whether the spatiotemporal deviation of each segment is high. The numerical range of this spatiotemporal deviation anomaly threshold is 0-1, and the specific value is set according to the actual judgment needs; as an example, a value of 0.5 is used. Flight segments with a spatiotemporal deviation greater than the spatiotemporal deviation anomaly threshold are determined as abnormal flight segments. If there are abnormal flight segments, there can be only one or multiple abnormal flight segments. If there are no abnormal flight segments, the subsequent step S6 is not executed.

[0069] Step S6: Adjust the large-scale programmable model of the abnormal segments to indicate the segment information of the extracted abnormal segments for reprogramming.

[0070] For any abnormal flight segment, the large programmable model of the abnormal flight segment is adjusted. To improve the accuracy of segment information extraction, the large programmable model of each field of the abnormal flight segment is adjusted to a deep inference model. The deep inference model is then called on each field of the abnormal flight segment for re-programming and extraction to re-extract the segment information of the abnormal flight segment.

[0071] In an exemplary embodiment, after the deep inference model is invoked to re-arrange and extract each field of the abnormal flight segment, the spatiotemporal deviation of each abnormal flight segment can be calculated again and compared with the spatiotemporal deviation anomaly threshold again. If it is less than or equal to the spatiotemporal deviation anomaly threshold, the adjustment result of the programmable large model is accepted and the verification is completed; otherwise, a high anomaly label is added to the abnormal flight segment and the corresponding alarm signal is output to facilitate the staff to find the cause of the anomaly.

[0072] This embodiment also provides a travel segment information extraction and verification system based on a programmable large model, including: a memory and a processor; the memory is connected to the processor, and the memory is used to store program instructions; the processor is used to implement the steps in the above embodiment of the travel segment information extraction and verification method based on a programmable large model when the program instructions are executed.

[0073] In one exemplary embodiment, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the embodiment of the travel segment information extraction and verification method based on a programmable large model.

[0074] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0075] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for extracting and verifying travel segment information based on a programmable large model, characterized in that, include: Extract the set of text blocks for each flight segment from multiple travel documents to be processed; Based on the occurrence of each field in the set of text blocks of each flight segment, the field uncertainty of each field in each flight segment is determined; Based on the uncertainty of the fields, a corresponding programmable large model is assigned to each field of each segment to indicate that each field of each segment is arranged according to the corresponding programmable large model and output the field target candidate value of each field of each segment. Based on the candidate values ​​of the field targets for each segment, the spatiotemporal deviation of each segment is determined; Abnormal flight segments are determined by the spatiotemporal deviation of each flight segment; Adjust the large programmable model of the abnormal flight segment to indicate the reprogramming extraction of the flight segment information of the abnormal flight segment.

2. The method for extracting and verifying travel segment information based on a programmable large model as described in claim 1, characterized in that, The step of extracting the set of flight segment text blocks for each flight segment from multiple travel documents to be processed includes: The text of each travel document to be processed is extracted to obtain the original text sequences. The original text sequence is segmented into text blocks to obtain flight segment text blocks; Group the text blocks of the same flight segment together to obtain the set of text blocks of each flight segment.

3. The method for extracting and verifying travel segment information based on a programmable large model as described in claim 1, characterized in that, The uncertainty in determining each field of each flight segment includes: Determine the text consistency of each field in each flight segment; the text consistency represents the situation where the same field value appears most frequently in the corresponding flight segment text block set; Determine the domain rule compliance degree of each field in each flight segment; the domain rule compliance degree represents the situation in which each field of each flight segment satisfies the corresponding field rule in the corresponding flight segment text block set; The field uncertainty is obtained based on the text consistency and domain rule compliance; the field uncertainty is inversely correlated with both the text consistency and domain rule compliance.

4. The method for extracting and verifying travel segment information based on a programmable large model as described in claim 3, characterized in that, The process of obtaining text consistency includes: Get the maximum number of occurrences of any field in any given flight segment within the corresponding set of flight segment text blocks; The ratio of the maximum occurrence count to the number of segment text blocks in the corresponding segment text block set is used as the text consistency of any field in any segment. The process of obtaining the domain rule conformity includes: Get the number of times a field in any given flight segment satisfies the corresponding field rule in the corresponding flight segment text block set; The ratio of the number of elements satisfying the corresponding field rule to the number of elements in the corresponding segment text block set is used as the domain rule compliance degree of any field of any segment.

5. The method for extracting and verifying travel segment information based on a programmable large model as described in claim 1, characterized in that, The programmable large model is divided into a lightweight extraction model, a standard understanding model, and a deep reasoning model; The assignment of corresponding programmable large models to each field of each flight segment based on field uncertainty includes: Compare the field uncertainty of any field in any flight segment with the magnitudes of a first threshold and a second threshold; the first threshold is greater than the second threshold. When the uncertainty of the field is less than the second threshold, a lightweight extraction model is assigned to any field of any flight segment; When the uncertainty of the field is less than the first threshold and greater than or equal to the second threshold, a standard understanding model is assigned to any field of any flight segment; When the uncertainty of the field is greater than or equal to the first threshold, a deep inference model is assigned to any field of any flight segment.

6. The method for extracting and verifying travel segment information based on a programmable large model as described in claim 5, characterized in that, When arranging numeric fields according to the deep inference model, the standard understanding model is also called to arrange them, and two candidate values ​​for the fields are output. The two candidate values ​​are then weighted and fused to obtain the corresponding target candidate values ​​for the fields.

7. The method for extracting and verifying travel segment information based on a programmable large model as described in claim 1, characterized in that, The determination of the spatiotemporal deviation of each flight segment based on the candidate values ​​of each field in each flight segment includes: Determine the time difference between the theoretical flight time and the actual flight time for any given flight segment; Determine the spatial continuity indicator variable between any flight segment and its adjacent next flight segment; wherein, if the distance between the arrival point of any flight segment and the departure point of its adjacent next flight segment is greater than a preset distance, the spatial continuity indicator variable is a first preset value, otherwise it is a second preset value, and the first preset value is greater than the second preset value. Determine the time continuity indicator variable between any flight segment and its adjacent next flight segment; wherein, if the transfer time between any flight segment and its adjacent next flight segment is less than a preset time, the time continuity indicator variable is the first preset value, otherwise it is the second preset value; By integrating the time difference, spatial continuity indicator variable, and time continuity indicator variable, the spatiotemporal deviation of any flight segment is obtained; the spatiotemporal deviation is positively correlated with the time difference, spatial continuity indicator variable, and time continuity indicator variable.

8. The method for extracting and verifying travel segment information based on a programmable large model as described in claim 1, characterized in that, The determination of abnormal flight segments based on the spatiotemporal deviation of each flight segment includes: Flight segments with a spatiotemporal deviation greater than the spatiotemporal deviation anomaly threshold are identified as the abnormal flight segments.

9. The method for extracting and verifying travel segment information based on a programmable large model as described in claim 5, characterized in that, The programmable large model for adjusting the abnormal flight segments includes: The programmable large model of each field of the abnormal flight segment is adjusted to a deep inference model.

10. A travel segment information extraction and verification system based on a programmable large model, characterized in that it includes: Memory and processor; The memory is connected to the processor; The memory is used to store program instructions; The processor is configured to implement, when program instructions are executed, the travel segment information extraction and verification method based on a programmable large model as described in any one of claims 1-9.

Images