Airport guarantee evaluation method and system based on large language model

Abstract

The invention discloses an airport guarantee evaluation method and system based on a large language model, and the method comprises the steps: carrying out the preprocessing of the obtained to-be-classified text data, so as to obtain the first type of text data; performing text segmentation and dimension evaluation on the first type of text data based on a large language model and a predefined evaluation dimension to obtain a plurality of text segments; performing sentiment analysis on the text segments in each evaluation dimension one by one based on a large language model to obtain sentiment tendencies corresponding to the plurality of text segments one by one, the sentiment tendencies including positive, negative and neutral; calculating weights and scores corresponding to the plurality of evaluation dimensions based on the emotional tendencies corresponding to the plurality of text segments one by one and the plurality of text segments; and according to the weights of the plurality of evaluation dimensions corresponding to the plurality of text segments and the scores, calculating a total score of entity services to obtain a corresponding evaluation result. According to the method and the system, the evaluation efficiency and objectivity are improved, and the complexity of data collection is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of civil aviation service capability assessment, and in particular to an airport support assessment method and system based on a large language model. Background Art

[0002] Traditional civil aviation service capability assessment methods mainly rely on questionnaires and Likert scales. These methods evaluate service quality by obtaining feedback directly from passengers. Although they are effective to a certain extent, they also have obvious limitations. Questionnaires usually require a lot of manpower and material resources to design and implement. Due to the limitations of sample size and survey coverage, it is difficult to fully reflect the actual situation. At the same time, the scoring criteria of the Likert scale may be too subjective and difficult to capture the specific feelings of passengers on service details. With the rapid development of the Internet and social media, a large amount of user-generated content has been generated on platforms such as Weibo, Xiaohongshu, Toutiao, and Douyin. These text reviews are huge in number and rich in content, covering multiple dimensions of service, such as safety, comfort, punctuality, customer service, etc. Traditional analysis methods often require a lot of training and annotation work when processing these unstructured data, which is costly. Especially when adding new categories, a lot of manpower and material resources are needed to rebuild the model and annotate the data, resulting in a lot of manpower and material resources required for the evaluation work. Summary of the invention

[0003] In order to solve the technical problems existing in the background technology, the present invention proposes an airport security assessment method and system based on a large language model.

[0004] The present invention proposes an airport security assessment method based on a large language model, comprising the following steps:

[0005] S1, preprocessing the acquired text data to be classified to obtain first type of text data;

[0006] S2, performing text segmentation and evaluation on the first type of text data based on the large language model and predefined evaluation dimensions to obtain multiple text segments;

[0007] S3, based on the large language model, sentiment analysis is performed on the text segments in each evaluation dimension one by one to obtain sentiment tendencies corresponding to the multiple text segments, wherein the sentiment tendencies include positive, negative, and neutral;

[0008] S4, calculating weights and scores corresponding to multiple evaluation dimensions based on the sentiment tendencies corresponding to the multiple text segments and the multiple text segments;

[0009] S5. Calculate the total score of the entity service according to the weights and scores of multiple evaluation dimensions corresponding to the multiple text fragments to obtain the corresponding evaluation result.

[0010] Preferably, the predefined evaluation dimensions include but are not limited to airport commerce, airport transportation, airport security, flight irregularity protection, baggage services, and airport services and facilities.

[0011] Preferably, the preprocessing specifically includes:

[0012] Noise and redundant information are removed from the text data to be classified, and the text data to be classified after the noise and redundant information are removed is used as the first type of text data.

[0013] Preferably, step S2 specifically includes:

[0014] The first type of text data is segmented based on the large language model, the first type of text data is split into text segments related to each evaluation dimension, and each value segment reflecting one evaluation dimension is retained to obtain multiple text segments.

[0015] Preferably, the weight calculation process corresponding to the multiple evaluation dimensions is as follows:

[0016] Eliminate text segments with neutral sentiment in each evaluation dimension to obtain the total number x of remaining text segments corresponding to multiple evaluation dimensions;

[0017] After standardizing the data corresponding to multiple evaluation dimensions based on the total number x of remaining text fragments, the indicator information entropy of each evaluation dimension is calculated;

[0018] The weight of each evaluation dimension is calculated based on the indicator information entropy of each evaluation dimension.

[0019] Preferably, the indicator information entropy is specifically:

[0020]

[0021] in, is the number of evaluation entities, if p ij =0, then define

[0022] Preferably, the weight of each evaluation dimension is calculated based on the indicator information entropy of each evaluation dimension, specifically including:

[0023] Calculate the difference coefficient of each evaluation dimension based on the indicator information entropy of each evaluation dimension;

[0024] The weight of each evaluation dimension is calculated based on the difference coefficient of each evaluation dimension;

[0025] The difference coefficient of each evaluation dimension is specifically:

[0026] g j =1-ej ;

[0027] The weight of each evaluation dimension is specifically:

[0028]

[0029] Among them, w j is the weight corresponding to the jth evaluation dimension; g j is the coefficient of difference corresponding to the j-th evaluation dimension.

[0030] Preferably, the score calculation process corresponding to the multiple evaluation dimensions is as follows:

[0031] Eliminate text segments with neutral sentiment in each evaluation dimension to obtain the total number of text segments, the number of positive text segments, and the number of negative text segments corresponding to each evaluation dimension;

[0032] The score corresponding to each evaluation dimension is calculated based on the total number of text fragments, the number of positive text fragments, and the number of negative text fragments corresponding to each evaluation dimension.

[0033] Preferably, the total score of the entity service is calculated according to the weights and scores of the multiple evaluation dimensions corresponding to the multiple text segments, specifically:

[0034]

[0035] Among them, S i represents the evaluation dimension score; w i represents the weight of the evaluation dimension; n represents the number of evaluation dimensions.

[0036] The present invention proposes an airport security assessment system based on a large language model, comprising:

[0037] A data acquisition module, used for preprocessing the acquired text data to be classified to obtain first type text data;

[0038] A first processing module, configured to perform text segmentation and evaluation on the first type of text data based on the large language model and predefined evaluation dimensions to obtain a plurality of text segments;

[0039] The second processing module is used to perform sentiment analysis on the text segments in each evaluation dimension one by one based on the large language model to obtain sentiment tendencies corresponding to the multiple text segments, wherein the sentiment tendencies include positive, negative, and neutral;

[0040] A third processing module is used to calculate weights and scores corresponding to multiple evaluation dimensions based on the sentiment tendencies corresponding to the multiple text segments and the multiple text segments;

[0041] The data output module is used to calculate the total score of the entity service according to the weights and scores of multiple evaluation dimensions corresponding to multiple text fragments to obtain the corresponding evaluation results.

[0042] In the present invention, the proposed airport support evaluation method and system based on a large language model pre-processes the acquired text data to be classified to obtain the first type of text data; based on the large language model and the pre-defined evaluation dimension, the first type of text data is segmented and evaluated to obtain multiple text segments; based on the large language model, the text segments in each evaluation dimension are sentimentally analyzed one by one to obtain the sentiment tendencies corresponding to the multiple text segments, and the sentiment tendencies include positive, negative, and neutral; based on the sentiment tendencies corresponding to the multiple text segments and the multiple text segments, the weights and scores corresponding to the multiple evaluation dimensions are calculated; according to the weights and scores of the multiple evaluation dimensions corresponding to the multiple text segments, the total score of the entity service is calculated to obtain the corresponding evaluation result. The large language model is used to automatically process the massive text data generated by users, and multi-dimensional emotions and opinions are extracted from them, providing a more efficient and accurate service capability evaluation than the traditional method, thereby improving the evaluation efficiency and accuracy. By reducing the dependence on data training and annotation, especially when new classification requirements are added, the manpower and material costs can be significantly reduced. It can quickly adapt to different service dimensions and classification requirements, and provide a flexible evaluation scheme to cope with the ever-changing market environment and user expectations. Through multi-dimensional analysis, it fully reflects users' real feedback on civil aviation services, improves the efficiency and objectivity of the evaluation, reduces the complexity of data collection, and improves data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of the workflow structure of an airport security assessment method based on a large language model proposed by the present invention;

[0044] Figure 2 This is a schematic diagram of the system architecture of an airport security assessment system based on a large language model proposed by the present invention. DETAILED DESCRIPTION

[0045] Reference Figure 1 The present invention proposes an airport security assessment method based on a large language model, comprising the following steps:

[0046] S1. Preprocess the acquired text data to be classified to obtain first type text data.

[0047] In this embodiment, the predefined evaluation dimensions include, but are not limited to, airport commerce, airport transportation, airport security, flight irregularity protection, baggage services, and airport services and facilities.

[0048] In this embodiment, the preprocessing specifically includes:

[0049] Noise and redundant information are removed from the text data to be classified, and the text data to be classified after the noise and redundant information are removed is used as the first type of text data.

[0050] S2. Performing text segmentation and evaluation on the first type of text data based on the large language model and predefined evaluation dimensions to obtain a plurality of text segments.

[0051] In this embodiment, step S2 specifically includes:

[0052] The first type of text data is segmented based on the large language model, the first type of text data is split into text segments related to each evaluation dimension, and each value segment reflecting one evaluation dimension is retained to obtain multiple text segments.

[0053] S3. Based on the large language model, sentiment analysis is performed on the text segments in each evaluation dimension one by one to obtain the sentiment tendencies corresponding to multiple text segments, and the sentiment tendencies include positive, negative, and neutral.

[0054] In this embodiment, the large language model is used to define the learning of indicator classification, natural language reasoning ability, and zero-sample classification ability, and the text is segmented to split the long text into short text fragments related to each dimension, retaining each valuable fragment reflecting the information of one dimension. The large language model is used to classify the text fragments and classify them into pre-defined evaluation dimensions to ensure the accuracy and consistency of the classification. The text fragments within each dimension are subjected to sentiment analysis to identify positive, negative, or neutral sentiment tendencies. The sentiment analysis capability of the large language model is used to ensure the accuracy of sentiment classification. For example: "When I bought the ticket, the airline said there would be meals, but it turned into a bag of peanuts. There was no explanation. The seats were super crowded, and the small table was put down directly on my stomach. I waited a full hour and 2 minutes from the time of arrival to check in." Output: [["A01","When I bought the ticket, the airline said there would be meals, but it turned into a bag of peanuts. There was no explanation","negative"],["A06","The seats were super crowded, and the small table was put down directly on my stomach","negative"],["A08","I waited a full hour and 2 minutes from the time of arrival to check in","negative"]].

[0055] S4. Calculate weights and scores corresponding to multiple evaluation dimensions based on the one-to-one corresponding sentiment tendencies of multiple text fragments and the multiple text fragments.

[0056] In this embodiment, the weight calculation process corresponding to multiple evaluation dimensions is as follows: eliminate text segments with neutral sentiment in each evaluation dimension to obtain the total number x of remaining text segments corresponding to multiple evaluation dimensions; after standardizing the data corresponding to multiple evaluation dimensions based on the total number x of remaining text segments, calculate the indicator information entropy of each evaluation dimension; calculate the weight of each evaluation dimension based on the indicator information entropy of each evaluation dimension.

[0057] Specifically, the indicator information entropy is:

[0058]

[0059] in, is the number of evaluation entities, if p ij =0, then define

[0060] In this embodiment, the difference coefficient of each evaluation dimension is calculated according to the indicator information entropy of each evaluation dimension; and the weight of each evaluation dimension is calculated according to the difference coefficient of each evaluation dimension.

[0061] Specifically, the difference coefficient of each evaluation dimension is:

[0062] g j =1-e j ;

[0063] Specifically, the weight of each evaluation dimension is:

[0064]

[0065] Among them, w j is the weight corresponding to the jth evaluation dimension; g j is the coefficient of difference corresponding to the j-th evaluation dimension.

[0066] In this embodiment, the score calculation process corresponding to multiple evaluation dimensions is as follows: eliminate text fragments with neutral sentiment in each evaluation dimension to obtain the total number of text fragments, the number of positive text fragments, and the number of negative text fragments corresponding to each evaluation dimension; calculate the score corresponding to each evaluation dimension based on the total number of text fragments, the number of positive text fragments, and the number of negative text fragments corresponding to each evaluation dimension.

[0067] Specifically, for each evaluation dimension, the favorable rate is first calculated. The favorable rate is the number of positive segments divided by the total number of text segments (positive segments + negative segments). If the number of negative segments is 0, the total number of text segments = the number of positive segments, and the favorable rate is 1. P is the number of positive segments, and N is the number of negative segments.

[0068] Compute the lower bound of the Wilson interval:

[0069]

[0070] in, is the rate of favorable comments on this indicator for this entity, n is the total number of text fragments (positive + negative) for this evaluation dimension of this entity, and z is the z-score corresponding to the required confidence level (the z-score corresponding to the 95% confidence level is approximately 1.96).

[0071] Calculate the indicator score:

[0072] s=1+4*p min ;

[0073] Use 5-point evaluation score.

[0074] In this embodiment, the data of each evaluation dimension is standardized to make the data dimensionless and easy to compare. The standardized formula can be:

[0075]

[0076] Among them, y ij is the standardized value; x ij is the original data; min(x j ) and max(x j ) are the minimum and maximum values ​​of the j-th indicator respectively.

[0077] S5. Calculate the total score of the entity service according to the weights and scores of multiple evaluation dimensions corresponding to the multiple text fragments to obtain the corresponding evaluation result.

[0078] In this embodiment, the total service score of the entity is calculated as follows:

[0079]

[0080] Among them, S i represents the evaluation dimension score; w i represents the weight of the evaluation dimension; n represents the number of evaluation dimensions.

[0081] Reference Figure 1 and Figure 2 The present invention proposes an airport security assessment system based on a large language model, comprising:

[0082] A data acquisition module, used for preprocessing the acquired text data to be classified to obtain first type text data;

[0083] A first processing module, configured to perform text segmentation and evaluation on the first type of text data based on the large language model and predefined evaluation dimensions to obtain a plurality of text segments;

[0084] The second processing module is used to perform sentiment analysis on the text segments in each evaluation dimension one by one based on the large language model to obtain sentiment tendencies corresponding to the multiple text segments, and the sentiment tendencies include positive, negative, and neutral;

[0085] A third processing module is used to calculate weights and scores corresponding to multiple evaluation dimensions based on the sentiment tendencies corresponding to the multiple text segments and the multiple text segments;

[0086] The data output module is used to calculate the total score of the entity service according to the weights and scores of multiple evaluation dimensions corresponding to multiple text fragments to obtain the corresponding evaluation results.

[0087] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for airport security assessment based on a large language model, characterized in that: The following steps are involved: S1, preprocessing the acquired text data to be classified to obtain first type of text data; S2, performing text segmentation and evaluation on the first type of text data based on the large language model and predefined evaluation dimensions to obtain multiple text segments; S3, based on the large language model, sentiment analysis is performed on the text segments in each evaluation dimension one by one to obtain sentiment tendencies corresponding to the multiple text segments, wherein the sentiment tendencies include positive, negative, and neutral; S4, calculating weights and scores corresponding to multiple evaluation dimensions based on the sentiment tendencies corresponding to the multiple text segments and the multiple text segments; S5. Calculate the total score of the entity service according to the weights and scores of multiple evaluation dimensions corresponding to the multiple text fragments to obtain the corresponding evaluation result.

2. The airport security assessment method based on a large language model according to claim 1 is characterized in that: The predefined evaluation dimensions include, but are not limited to, airport commerce, airport transportation, airport security, flight irregularity protection, baggage services, and airport services and facilities.

3. The airport security assessment method based on a large language model according to claim 1 is characterized in that: The pre-processing specifically includes: Noise and redundant information are removed from the text data to be classified, and the text data to be classified after the noise and redundant information are removed is used as the first type of text data.

4. The airport security assessment method based on a large language model according to claim 2 is characterized in that: Step S2 specifically includes: The first type of text data is segmented based on the large language model, the first type of text data is split into text segments related to each evaluation dimension, and each value segment reflecting one evaluation dimension is retained to obtain multiple text segments.

5. The airport security assessment method based on a large language model according to claim 2 is characterized in that: The weight calculation process corresponding to the multiple evaluation dimensions is as follows: Eliminate text segments with neutral sentiment in each evaluation dimension to obtain the total number x of remaining text segments corresponding to multiple evaluation dimensions; After standardizing the data corresponding to multiple evaluation dimensions based on the total number x of remaining text fragments, the indicator information entropy of each evaluation dimension is calculated; The weight of each evaluation dimension is calculated based on the indicator information entropy of each evaluation dimension.

6. The airport security assessment method based on a large language model according to claim 5 is characterized in that: The indicator information entropy is specifically: in, is the number of evaluation entities, if p ij =0, then define 7. The airport security assessment method based on a large language model according to claim 6 is characterized in that: The weight of each evaluation dimension is calculated based on the indicator information entropy of each evaluation dimension, specifically including: Calculate the difference coefficient of each evaluation dimension based on the indicator information entropy of each evaluation dimension; The weight of each evaluation dimension is calculated based on the difference coefficient of each evaluation dimension; The difference coefficient of each evaluation dimension is specifically: g j =1-e j ; The weight of each evaluation dimension is specifically: Among them, w j is the weight corresponding to the jth evaluation dimension; g j is the coefficient of difference corresponding to the j-th evaluation dimension.

8. The airport security assessment method based on a large language model according to claim 7 is characterized in that: The score calculation process corresponding to the multiple evaluation dimensions is as follows: Eliminate text segments with neutral sentiment in each evaluation dimension to obtain the total number of text segments, the number of positive text segments, and the number of negative text segments corresponding to each evaluation dimension; The score corresponding to each evaluation dimension is calculated based on the total number of text fragments, the number of positive text fragments, and the number of negative text fragments corresponding to each evaluation dimension.

9. The airport security assessment method based on a large language model according to claim 8 is characterized in that: The total score of the entity service is calculated according to the weights and scores of the multiple evaluation dimensions corresponding to the multiple text segments, specifically: Among them, S i represents the evaluation dimension score; w i represents the weight of the evaluation dimension; n represents the number of evaluation dimensions.

10. An airport support assessment system based on a large language model, characterized in that: include: A data acquisition module, used for preprocessing the acquired text data to be classified to obtain first type text data; A first processing module, configured to perform text segmentation and evaluation on the first type of text data based on the large language model and predefined evaluation dimensions to obtain a plurality of text segments; The second processing module is used to perform sentiment analysis on the text segments in each evaluation dimension one by one based on the large language model to obtain sentiment tendencies corresponding to the multiple text segments, wherein the sentiment tendencies include positive, negative, and neutral; A third processing module is used to calculate weights and scores corresponding to multiple evaluation dimensions based on the sentiment tendencies corresponding to the multiple text segments and the multiple text segments; The data output module is used to calculate the total score of the entity service according to the weights and scores of multiple evaluation dimensions corresponding to multiple text fragments to obtain the corresponding evaluation results.

Images