Civil aviation freight rate anomaly detection-oriented feature screening method and system and electronic equipment

Abstract

The invention relates to the technical field of civil aviation information processing, and provides a civil aviation freight rate anomaly detection-oriented feature screening method and system and electronic equipment, and the method comprises the steps: carrying out the data cleaning of an original freight rate calculation log file; screening all effective information having significant influence on the civil aviation freight rate anomaly detection task from the cleaned freight rate calculation log file to form an effective information set, and extracting feature vectors; constructing a feature vector based on a construction rule provided by a business expert and business features influencing the civil aviation freight rate in a civil aviation business scene; and carrying out validity verification on the extracted feature vector and the constructed feature vector. According to the method, the key features influencing the civil aviation freight rate can be accurately and efficiently screened out, the data feature vector with high-value semantic information is provided for a civil aviation anomaly detection task, and the precision and reliability of subsequent algorithm analysis are improved.

Description

Technical Field

[0001] The present application belongs to the field of civil aviation information processing technology, and in particular relates to a feature screening method, system and electronic equipment for detecting abnormal civil aviation fares. Background Art

[0002] With the rapid development of the aviation industry, the management and analysis of civil aviation fare data has become particularly important. The current civil aviation fare data contains a large amount of information, and the features contained in this information play a key role in anomaly detection, price prediction, market analysis, and competitive strategy formulation.

[0003] The existing civil aviation freight rate data has the characteristics of complex data form, which is specifically reflected in the following aspects:

[0004] 1) Diversification of static attributes: Freight data contains a large number of attributes, such as flight information, sales channels, fares, passenger information, cabin class, departure point, destination point and flight time, etc. The data formats, categories and meanings of these attributes vary, which increases the complexity of data processing and selection.

[0005] 2) Dynamic changes over time: Air ticket prices change dynamically over time, for example, they are affected by factors such as seasonal fluctuations, holiday premiums, and booking lead time.

[0006] 3) Special event disturbances: Freight rate data is easily disturbed by special events such as weather changes, policy adjustments, and emergencies, leading to price fluctuations.

[0007] 4) Diversified patterns: The freight rate data of different routes have different patterns. Especially in the peak season, the dynamic changes in passenger flow direction may lead to opposite price trends on different routes.

[0008] Existing traditional technologies face many challenges when processing these civil aviation fare data. For example, the task of detecting anomalies in civil aviation fare requires accurate extraction of data features that are highly correlated with pricing factors, and generates feature vectors for each product of civil aviation fare. However, there is currently no good feature screening method that can accurately and efficiently extract data features that are highly correlated with pricing factors from civil aviation fare data. Summary of the invention

[0009] In response to the above-mentioned problems, the present application provides a feature screening method, system and electronic equipment for detecting abnormal civil aviation fares.

[0010] According to a first aspect of the present application, a feature screening method for detecting abnormal civil aviation fares is provided, the method comprising:

[0011] Perform data cleaning on the original freight rate calculation log file;

[0012] Filter out all valid information that has a significant impact on the civil aviation fare anomaly detection task from the cleaned fare calculation log file, form a valid information set, and extract feature vectors;

[0013] Construct feature vectors based on the construction rules provided by business experts and the business features that affect civil aviation fares in civil aviation business scenarios;

[0014] The validity of the extracted feature vectors and constructed feature vectors is verified.

[0015] Optionally, the data cleaning of the original freight rate calculation log file includes:

[0016] Read the original freight rate calculation log file, filter and sort it, and retain the fields in the original freight rate calculation log file that are of practical significance to civil aviation freight rate calculation;

[0017] Check each field to identify and handle missing and duplicate values;

[0018] Frequency encoding is performed on discrete data to convert discrete data into continuous numerical data.

[0019] Optionally, constructing a feature vector based on construction rules provided by business experts and business features affecting civil aviation fares in civil aviation business scenarios includes:

[0020] Carry out business scenario modeling, determine the business features that affect civil aviation freight rates from the cleaned freight rate calculation log files, and form a business feature set;

[0021] Construct features using business features in a business feature set according to construction rules provided by business experts;

[0022] Combine the constructed features into a feature vector.

[0023] Optionally, the validating the extracted feature vector and the constructed feature vector includes:

[0024] The explanatory power analysis of the extracted eigenvectors was performed, and the eigenvariables with cumulative variance contribution rate reaching 90% were taken as the final extracted eigenvectors;

[0025] The anomaly detection model based on Gaussian probability distribution verifies the validity of the final extracted feature vector and the constructed feature vector.

[0026] Optionally, all valid information that has a significant impact on the civil aviation fare anomaly detection task includes air ticket price, take-off time, landing time, booking time, passenger type, origin and destination.

[0027] Optionally, principal component analysis is used to extract feature vectors from the effective information set.

[0028] Optionally, the missing values ​​include continuous missing data and discrete missing data; and the identifying and processing missing values ​​and repeated values ​​includes:

[0029] Remove duplicate values ​​and discrete missing data;

[0030] Fill continuous missing data using the mean.

[0031] Optionally, the Gaussian probability distribution-based anomaly detection model verifies the validity of the finally extracted feature vector and the constructed feature vector, including:

[0032] Data preparation: The final extracted feature vector and the constructed feature vector are taken as input;

[0033] Establish anomaly detection model with Gaussian probability distribution;

[0034] Set the abnormal probability threshold and calculate the model evaluation index;

[0035] Feedback iteration and optimization: Repeatedly test the performance of the anomaly detection model under different parameters, use the difference in the performance of the anomaly detection model on different feature sets to verify the effectiveness of feature selection, extraction and construction, and output feature vectors and corresponding evaluation indicators.

[0036] According to a second aspect of the present application, a feature screening system for detecting abnormal civil aviation fares is provided, the feature screening system comprising:

[0037] The data cleaning module can clean the original freight rate calculation log file;

[0038] The feature selection and extraction module can filter out all valid information that has a significant impact on the civil aviation freight rate anomaly detection task from the cleaned freight rate calculation log file, form a valid information set, and extract feature vectors;

[0039] The feature construction module can construct feature vectors based on the construction rules provided by business experts and the business features that affect civil aviation fares in civil aviation business scenarios;

[0040] The feature analysis module can verify the feature validity of the extracted feature vectors and the constructed feature vectors.

[0041] According to a third aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0042] Beneficial technical effects:

[0043] 1. Aiming at the complex morphology of civil aviation fare log data, this application uses advanced machine learning algorithms and data mining methods, starting from data cleaning, feature selection and extraction, feature construction, and feature analysis, and systematically proposes a feature screening method and system for civil aviation fare anomaly detection. The method aims to systematically process civil aviation fare data and accurately and efficiently screen out key features that affect civil aviation fares, provide data feature vectors with high-value semantic information for civil aviation anomaly detection tasks, and improve the accuracy and reliability of subsequent algorithm analysis.

[0044] 2. This application can reduce the dimension of the feature vector, and use the feature vector obtained by screening in this application as the input of the subsequent abnormal freight rate detection algorithm, which can achieve high-precision identification of abnormal prices and help reduce the computing power and storage overhead of the detection system.

[0045] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures indicated in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 A flowchart of the steps of a feature screening method for detecting abnormal civil aviation fares according to an embodiment of the present application is shown.

[0048] Figure 2 A flow chart of the steps for data cleaning in an embodiment of the present application is shown.

[0049] Figure 3 The source of the original freight rate calculation log file in the embodiment of the present application is shown.

[0050] Figure 4 A flow chart of the steps for constructing a feature vector in an embodiment of the present application is shown.

[0051] Figure 5 A flow chart of the steps for performing feature analysis in an embodiment of the present application is shown.

[0052] Figure 6 A flow chart of the steps for performing validity verification in an embodiment of the present application is shown.

[0053] Figure 7 An example diagram of the architecture of a feature screening system for detecting abnormal civil aviation fares according to an embodiment of the present application is shown.

[0054] Figure 8 An exemplary diagram of the architecture of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0057] Figure 1 The following is a flowchart showing the steps of the feature screening method for detecting abnormal civil aviation fares according to an embodiment of the present application. Figure 1 As shown, the feature screening system may include the following steps S1 to S4.

[0058] S1: Data cleaning of the original freight rate calculation log file. This step is the basic step of the entire feature screening project.

[0059] Among them, Figure 2 As shown, the data cleaning of the original freight rate calculation log file may include the following steps S11 to S13.

[0060] S11: Read the original freight rate calculation log file, filter and sort it, and retain the fields in the original freight rate calculation log file that are of practical significance to the civil aviation freight rate calculation.

[0061] It should be noted that the freight rate data for this application comes from the original freight rate calculation log file. Figure 3As shown, after receiving an external pricing request, the calculation engine of the civil aviation pricing system will perform real-time freight calculations based on the pricing rules and related data in the freight database, and will generate transaction-related log records; freight data can be obtained by parsing and pushing historical or real-time transaction logs. Therefore, the source of the freight data of this application is the freight data, that is, the original freight calculation log file.

[0062] In some embodiments of the present application, the original fare calculation log files can be read offline or online. These log files contain a large number of transaction records, each record including multiple fields, such as ticket price, departure place, destination, passenger type, airline, etc.

[0063] After reading the original fare calculation log file, it is screened and sorted through pattern matching, deleting fields in the original fare calculation log file that have no practical significance for civil aviation fare calculation, such as system identifiers and prompt information, while retaining fields in the original fare calculation log file that have practical significance for civil aviation fare calculation, such as flight starting point, start and end time, and ticket price.

[0064] S12: Check each field, identify and process missing and duplicate values ​​to ensure the accuracy and completeness of the data.

[0065] In some embodiments of the present application, the missing values ​​include continuous missing data and discrete missing data; wherein, continuous missing data include ticket prices and departure times, and discrete missing data include flight classes and route origins. The identification and processing of missing values ​​and duplicate values ​​include: deleting duplicate values ​​and discrete missing data; filling continuous missing data with the mean. By deleting duplicate values, the uniqueness and accuracy of the data can be ensured. For discrete missing data, due to its criticality and particularity, implementing system abnormal output and screening out the freight rate data can avoid adverse effects on the overall quality.

[0066] S13: Perform frequency encoding on the discrete data to convert the discrete data into continuous numerical data.

[0067] In some embodiments of the present application, the cleaned discrete data is uniformly encoded, and a frequency encoding method is used to retain the frequency information of the discrete data for subsequent data analysis and feature extraction. The frequency encoding method is as follows:

[0068] First, let the input data set be D, which contains different categories of discrete variable C.

[0069] Next, calculate the number of category occurrences and the total number of samples: For each category c i ∈C, calculate the number of times it appears in the data set, recorded as count(c i). Assume the total number of samples is N, that is, N = |D|.

[0070] Then calculate the frequency: for each category c i , calculate the frequency

[0071] Finally, replace the original category value: each category c in the dataset i Replace it with its corresponding frequency value f(c i ).

[0072] S2: Filter out all valid information that has a significant impact on the civil aviation fare anomaly detection task from the cleaned fare calculation log file, form a valid information set, and extract the feature vector.

[0073] In some embodiments of the present application, the valid information includes information such as ticket price, departure time, landing time, booking time, passenger type, origin and destination, etc. In addition, features with semantic information such as flight distance, flight duration and holidays should also be considered.

[0074] In some embodiments of the present application, step S2 uses principal component analysis (PCA) to extract feature vectors from the effective information set to improve the performance of the final freight rate anomaly detection model. Specifically, the effective information in each cleaned log file is spliced ​​into an information vector, and the vectors generated by multiple log files are spliced ​​into an information matrix, and linear transformation is performed to a new space so that the variance of the data in each dimension of the new space is maximized, so that each of the extracted feature vectors can represent the distribution of the original data to the greatest extent. The specific steps are as follows:

[0075] First, generate the information matrix: Assuming that there are m elements in the valid information set, for each log file, generate an m-dimensional information vector (row vector), and concatenate the information vectors of n log files to generate the information matrix X n×m , where n should be much larger than m to improve the representativeness of the extracted features.

[0076] Second, the information matrix is ​​standardized. The conversion formula is:

[0077]

[0078] Among them, μ j and σ j are the mean and standard deviation of the j-th feature, and Z is the standardized data.

[0079] Third, calculate the sample covariance matrix C. The calculation formula is:

[0080]

[0081] Fourth, eigenvalue decomposition: calculate the eigenvalues ​​and eigenvectors of the covariance matrix C; the calculation formula is:

[0082] Cv i =λ i v i , i=1,...,m

[0083] Among them, λ i is the i-th eigenvalue, v i is the corresponding feature vector.

[0084] Fifth, select the principal components: sort the eigenvalues ​​in descending order and select the first k eigenvectors v1...,v k As the principal component, k should be smaller than m.

[0085] Sixth, data linear transformation: linearly project the original data into the new principal component space. The transformation formula is:

[0086] Y=ZV k

[0087] Among them, V k =[v1, ..., v k ], Y is the new feature matrix after conversion, where each i-th row represents the feature vector corresponding to the i-th log file.

[0088] S3: Construct feature vectors based on the construction rules provided by business experts and the business characteristics that affect civil aviation fares in civil aviation business scenarios.

[0089] Among them, Figure 4 As shown, the feature vector constructed based on the construction rules provided by the business experts and the business features that affect the civil aviation fare in the civil aviation business scenario includes:

[0090] S31: Business scenario modeling is performed to determine the business features that affect civil aviation fares from the cleaned fare calculation log file to form a business feature set. The business features include departure time, landing time, booking time, destination, etc.

[0091] S32: Construct features using business features in the business feature set according to construction rules provided by business experts.

[0092] In one embodiment of the present application, the construction rule provided by the business expert is: red-eye flights generally refer to flights with a departure time before 7:00 and a landing time after 22:00. According to the construction rule, the following features are constructed using the departure time and arrival time in the business feature set:

[0093] F Red-Eye =I(T departure <07:00)×I(Tarrival >22:00)

[0094] Among them, I (condition) is an indicator function. When the condition (i.e., condition) is established, I (condition) = 1; otherwise, I (condition) = 0. Red-Eye represents the red-eye flight characteristics, T departure and T arrival The departure time and arrival time respectively.

[0095] In one embodiment of the present application, the following features are constructed based on the construction rules and business features (distance and flight time) provided by business experts:

[0096]

[0097] Where D is the flight distance (in kilometers), T is the flight time (in hours), and F stopover It is the characteristic of a stopover flight.

[0098] In some embodiments of the present application, if individual sales of identification features, passenger identity features, and cabin availability are not allowed, they can be determined by identifying special fields in the log and using regular matching and other methods.

[0099] S33: Combine the constructed features into a feature vector.

[0100] In step S3 of the present application, for the civil aviation fare anomaly detection scenario, by analyzing the business features that have a greater impact on the fare in the civil aviation business scenario, new features are constructed based on this by matching the original feature dimension rules. For example, starting from simple basic features such as aircraft take-off time, landing time, and booking time, features with stronger semantic information and more direct correlation with air ticket pricing, such as red-eye flights, peak and off-peak seasons, and holidays, can be further extracted. By constructing feature vectors, the feature set can be enriched and expanded, the expressive power of the data can be enhanced, and the upper limit of the anomaly detection model can be further improved.

[0101] S4: Verify the validity of the extracted feature vectors and constructed feature vectors to ensure that they have a significant impact on civil aviation fares. Figure 5 As shown, the validity verification of the extracted feature vector and the constructed feature vector includes:

[0102] S41: Perform explanatory power analysis on the extracted eigenvectors, and take the eigenvariate whose cumulative variance contribution rate reaches 90% as the final extracted eigenvector.

[0103] Among them, the features extracted by principal component analysis (PCA) are first analyzed for their explanatory power. The eigenvalues ​​of the covariance matrix can be calculated during principal component analysis. The eigenvalues ​​reflect the size of the variance that each principal component can explain. Therefore, the size of the eigenvalues ​​can be used to judge the extraction effect of the principal component analysis method and set the number of principal components accordingly.

[0104] Specifically, when performing explanatory power analysis, the eigenvalue corresponding to each principal component is used as a reference indicator of explanatory power, and the relationship between the number of principal components and the overall explanatory power is examined to guide the determination of the final number of principal components. In the principal component analysis method, each eigenvalue corresponds to a principal component and reflects the degree to which the principal component explains the variance of the original data. The larger the eigenvalue, the more data variance the principal component can explain, and the stronger its explanatory power.

[0105] Therefore, the variance contribution of each principal component is:

[0106]

[0107] Among them, k is the number of principal components, λ i is the i-th eigenvalue.

[0108] Sort the eigenvalues ​​of all principal components and plot the relationship between the eigenvalues ​​and the number of principal components (i.e., the cumulative variance contribution rate graph). By observing this relationship graph, we can identify which principal components have stronger explanatory power and after which point, the newly added principal components contribute less to the explanatory power of the data. This application uses the number of principal components when the cumulative variance contribution rate reaches 90%.

[0109] In step S41 of the present application, by performing explanatory power analysis on the extracted feature vectors, the data dimension can be simplified while retaining the vast majority of information, thereby improving computational efficiency while ensuring analysis accuracy.

[0110] S42: The anomaly detection model based on Gaussian probability distribution verifies the validity of the final extracted feature vector and the constructed feature vector. Figure 6 As shown, the Gaussian probability distribution-based anomaly detection model verifies the validity of the finally extracted feature vector and the constructed feature vector, including:

[0111] S421: Data preparation: The final extracted feature vector and the constructed feature vector are taken as input.

[0112] S422: Establish an anomaly detection model based on Gaussian probability distribution. Specifically, the sample data is advanced

[0113] Among them, μ and σ are the sample mean and standard deviation within each category respectively.

[0114] S423: Set an abnormal probability threshold and calculate model evaluation indicators, which may include precision, recall rate, and F1 score.

[0115] S424: Feedback iteration and optimization: Repeatedly test the performance of the anomaly detection model under different parameters, use the difference in the performance of the anomaly detection model on different feature sets to verify the effectiveness of feature selection, extraction and construction, and output feature vectors and corresponding evaluation indicators.

[0116] In step S42 of the present application, the anomaly detection model based on Gaussian probability distribution fits normal data into a Gaussian normal distribution, and regards data outside the probability threshold as anomalies. The algorithm of the model is simple, so it can reflect the effectiveness of data feature selection, extraction and construction.

[0117] Based on the above-mentioned feature screening method, an embodiment of the present application provides a feature screening system for detecting abnormal civil aviation fares, which is explained below through the following embodiments.

[0118] The following describes a feature screening system for detecting abnormal civil aviation fares provided in an embodiment of the present application. The feature screening system described below and the feature screening method described above can be referenced to each other.

[0119] Figure 7 The following is an example diagram of the architecture of a feature screening system for detecting abnormal civil aviation fares according to an embodiment of the present application. Figure 7 As shown, the feature screening system includes a data cleaning module 20 , a feature selection and extraction module 30 , a feature construction module 40 and a feature analysis module 50 .

[0120] The data cleaning module 20 is used to clean the original freight rate calculation log file. The data cleaning of the original freight rate calculation log file includes: reading the original freight rate calculation log file, screening and sorting, retaining the fields in the original freight rate calculation log file that are of practical significance to the civil aviation freight rate calculation; checking each field, identifying and processing missing values ​​and duplicate values; frequency encoding discrete data, and converting discrete data into continuous numerical data. The input of the data cleaning module 20 is the original freight rate calculation log file, and the output is the cleaned log file.

[0121] The feature selection and extraction module 30 is used to filter out all valid information that has a significant impact on the civil aviation fare anomaly detection task from the cleaned fare calculation log file, form a valid information set, and extract feature vectors. The input of the feature selection and extraction module 30 is the cleaned log file, and the output is the extracted feature vector.

[0122] The feature construction module 40 is used to construct a feature vector based on the construction rules provided by the business experts and the business features that affect the civil aviation fares in the civil aviation business scenario; wherein, the construction of the feature vector based on the construction rules provided by the business experts and the business features that affect the civil aviation fares in the civil aviation business scenario includes: performing business scenario modeling, determining the business features that affect the civil aviation fares from the cleaned fare calculation log file, and forming a business feature set; constructing features using the business features in the business feature set according to the construction rules provided by the business experts; and combining the constructed features into a feature vector. The input of the feature construction module 40 is the cleaned log file, and the output is the constructed feature vector.

[0123] The feature analysis module 50 is used to verify the feature validity of the extracted feature vector and the constructed feature vector. The validity verification of the extracted feature vector and the constructed feature vector includes: performing explanatory power analysis on the extracted feature vector, taking the feature variable with a cumulative variance contribution rate of 90% as the final extracted feature vector; and verifying the validity of the final extracted feature vector and the constructed feature vector based on the anomaly detection model of Gaussian probability distribution. The input of the feature analysis module 50 is the extracted feature vector and the constructed feature vector, and the output is the feature vector and the evaluation index.

[0124] The feature screening method and system for abnormal detection of civil aviation fares of this application have the following beneficial effects:

[0125] 1. Aiming at the complex morphology of civil aviation fare log data, this application uses advanced machine learning algorithms and data mining methods, starting from data cleaning, feature selection and extraction, feature construction, and feature analysis, and systematically proposes a feature screening method and system for civil aviation fare anomaly detection. The method aims to systematically process civil aviation fare data and accurately and efficiently screen out key features that affect civil aviation fares, provide data feature vectors with high-value semantic information for civil aviation anomaly detection tasks, and improve the accuracy and reliability of subsequent algorithm analysis.

[0126] 2. This application can reduce the dimension of the feature vector, and use the feature vector obtained by screening in this application as the input of the subsequent abnormal freight rate detection algorithm, which can achieve high-precision identification of abnormal prices and help reduce the computing power and storage overhead of the detection system.

[0127] In addition, the feature screening method and system for civil aviation fare anomaly detection in this application solves the technical problem that there is currently no better feature screening method that can accurately and efficiently extract data features with large pricing correlation factors from civil aviation fare data, laying the foundation for the subsequent application of anomaly detection models in the field of civil aviation fares.

[0128] Figure 8FIG. 1 shows an example diagram of the architecture of an electronic device according to an embodiment of the present application. Figure 8 As shown, the electronic device includes a processor 110, a communication interface 120, a memory 130 for storing processor executable instructions, and a communication bus 140. The processor 110, the communication interface 120, and the memory 130 communicate with each other through the communication bus 140. The processor 50 implements the above-mentioned feature screening method by running an executable computer program.

[0129] Among them, the computer program in the memory 130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0130] The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected based on actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0132] Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that it is still possible to modify the technical solutions described in the aforementioned embodiments, or to make equivalent replacements for some of the technical features therein; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A feature screening method for abnormal detection of civil aviation freight rates, characterized in that: The method comprises: Perform data cleaning on the original freight rate calculation log file; Filter out all valid information that has a significant impact on the civil aviation fare anomaly detection task from the cleaned fare calculation log file, form a valid information set, and extract feature vectors; Construct feature vectors based on the construction rules provided by business experts and the business features that affect civil aviation fares in civil aviation business scenarios; The validity of the extracted feature vectors and constructed feature vectors is verified.

2. The feature screening method for abnormal detection of civil aviation fares according to claim 1 is characterized in that: The data cleaning of the original freight rate calculation log file includes: Read the original freight rate calculation log file, filter and sort it, and retain the fields in the original freight rate calculation log file that are of practical significance to civil aviation freight rate calculation; Check each field to identify and handle missing and duplicate values; Frequency encoding is performed on discrete data to convert discrete data into continuous numerical data.

3. The feature screening method for abnormal detection of civil aviation fares according to claim 1 is characterized in that: The feature vector constructed based on the construction rules provided by the business experts and the business features affecting the civil aviation fare in the civil aviation business scenario includes: Carry out business scenario modeling, determine the business features that affect civil aviation freight rates from the cleaned freight rate calculation log files, and form a business feature set; Construct features using business features in a business feature set according to construction rules provided by business experts; Combine the constructed features into a feature vector.

4. The feature screening method for abnormal detection of civil aviation fares according to claim 1 is characterized in that: The validity verification of the extracted feature vector and the constructed feature vector includes: The explanatory power analysis of the extracted eigenvectors was performed, and the eigenvariables with cumulative variance contribution rate reaching 90% were taken as the final extracted eigenvectors; The anomaly detection model based on Gaussian probability distribution verifies the validity of the final extracted feature vector and the constructed feature vector.

5. The feature screening method for abnormal detection of civil aviation fares according to claim 1 is characterized in that: All the valid information that has a significant impact on the civil aviation fare anomaly detection task includes air ticket price, take-off time, landing time, booking time, passenger type, origin and destination.

6. The feature screening method for abnormal detection of civil aviation fares according to claim 1 is characterized in that: The principal component analysis method is used to extract feature vectors from the effective information set.

7. The feature screening method for abnormal detection of civil aviation fares according to claim 2 is characterized in that: The missing values ​​include continuous missing data and discrete missing data; the identification and processing of missing values ​​and repeated values ​​includes: Remove duplicate values ​​and discrete missing data; Fill continuous missing data using the mean.

8. The feature screening method for detecting abnormal civil aviation fares according to claim 4, characterized in that: The Gaussian probability distribution-based anomaly detection model verifies the validity of the finally extracted feature vector and the constructed feature vector, including: Data preparation: The final extracted feature vector and the constructed feature vector are taken as input; Establish anomaly detection model with Gaussian probability distribution; Set the abnormal probability threshold and calculate the model evaluation index; Feedback iteration and optimization: Repeatedly test the performance of the anomaly detection model under different parameters, use the difference in the performance of the anomaly detection model on different feature sets to verify the effectiveness of feature selection, extraction and construction, and output feature vectors and corresponding evaluation indicators.

9. A feature screening system for abnormal detection of civil aviation freight rates, characterized in that: include: The data cleaning module can clean the original freight rate calculation log file; The feature selection and extraction module can filter out all valid information that has a significant impact on the civil aviation freight rate anomaly detection task from the cleaned freight rate calculation log file, form a valid information set, and extract feature vectors; The feature construction module can construct feature vectors based on the construction rules provided by business experts and the business features that affect civil aviation fares in civil aviation business scenarios; The feature analysis module can verify the feature validity of the extracted feature vectors and the constructed feature vectors.

10. An electronic device, characterized in that: The electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method according to any one of claims 1 to 8 when executing the program.

Images