Civil aviation fare anomaly detection-oriented feature screening method and system, and electronic device

Abstract

The application relates to the technical field of civil aviation information processing, and provides a feature screening method and system for civil aviation price anomaly detection and an electronic device, the method comprising the following steps: performing data cleaning on an original price calculation log file; screening all effective information which has a significant influence on a civil aviation price anomaly detection task from the cleaned price calculation log file, to form an effective information set, and extracting a feature vector; constructing the feature vector based on a construction rule provided by a business expert and a business feature which influences the civil aviation price in a civil aviation business scenario; and performing effectiveness verification on the extracted feature vector and the constructed feature vector. The application can accurately and efficiently screen out key features which influence the civil aviation price, provide a data feature vector with high-value semantic information for a civil aviation anomaly detection task, and improves the accuracy and reliability of subsequent algorithm analysis.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of civil aviation information processing, and particularly relates to a feature screening method and system for civil aviation price abnormality detection and an electronic device. BACKGROUND

[0002] With the rapid development of the aviation industry, the management and analysis of civil aviation price data have become particularly important. The current civil aviation price data contains a large amount of information, and the features contained in these information play a key role in abnormality detection, price prediction, market analysis, and competition strategy formulation.

[0003] The existing civil aviation price data has the characteristics of complex data form, which is specifically embodied in the following aspects:

[0004] 1) Multiple static attributes: The price data contains a large number of attributes, such as flight information, sales channels, ticket prices, passenger information, cabin grades, departure locations, destination locations, and flight times. The data formats, categories, and meanings of these attributes are different, increasing the complexity of data processing and selection.

[0005] 2) Dynamic changes over time: Ticket prices change dynamically over time, for example, influenced by seasonal fluctuations, holiday premiums, booking lead times, and other factors.

[0006] 3) Disturbance of special events: Price data is easily disturbed by weather changes, policy adjustments, and unexpected events, leading to price fluctuations.

[0007] 4) Variety of rules: The price data of different routes varies, especially in peak seasons, and the dynamic changes in passenger flow direction may cause the price trends of different routes to present opposite situations.

[0008] The existing traditional technology faces many challenges in processing these civil aviation price data. For example, in the civil aviation price abnormality detection task, it is required to accurately extract data features with a larger correlation factor with pricing and generate feature vectors for each product of civil aviation price. However, there is no good feature screening method to accurately and efficiently extract data features with a larger correlation factor with pricing from civil aviation price data. SUMMARY

[0009] The application provides a feature screening method, system, and electronic device for civil aviation price abnormality detection to solve the above problems.

[0010] According to a first aspect of the application, a feature screening method for civil aviation price abnormality detection is provided, which comprises:

[0011] performing data cleaning on the original price calculation log file;

[0012] Screen all valid information which has significant influence on the civil aviation price anomaly detection task from the cleaned freight calculation log file to form a valid information set, and extract a feature vector;

[0013] Construct the feature vector based on the construction rules provided by the business experts and the business features which affect the civil aviation price in the civil aviation business scenario;

[0014] Verify the effectiveness of the extracted feature vector and the constructed feature vector.

[0015] Optionally, the data cleaning on the original freight calculation log file comprises:

[0016] Read the original freight calculation log file, perform screening and arrangement, and retain the fields in the original freight calculation log file which have actual significance for civil aviation price calculation;

[0017] Check each field, identify and process missing values and duplicate values;

[0018] Frequency encoding is performed on the discrete data to convert the discrete data into continuous numerical data.

[0019] Optionally, the construction of the feature vector based on the construction rules provided by the business experts and the business features which affect the civil aviation price in the civil aviation business scenario comprises:

[0020] Perform business scenario modeling, determine the business features which affect the civil aviation price from the cleaned freight calculation log file, and form a business feature set;

[0021] According to the construction rules provided by the business experts, use the business features in the business feature set to construct features;

[0022] Combine the constructed features into a feature vector.

[0023] Optionally, the effectiveness verification on the extracted feature vector and the constructed feature vector comprises:

[0024] Perform explanatory power analysis on the extracted feature vector, and take the feature variables with a cumulative variance contribution rate of 90% as the final extracted feature vector;

[0025] The effectiveness of the final extracted feature vector and the constructed feature vector is verified based on a Gaussian probability distribution anomaly detection model.

[0026] Optionally, all the valid information which has significant influence on the civil aviation price anomaly detection task comprises ticket price, takeoff time, landing time, booking time, passenger type, origin and destination.

[0027] Optionally, the principal component analysis method is used to extract the feature vector from the valid information set.

[0028] Optionally, the missing values include continuous missing data and discrete missing data; and the identifying and processing of the missing values and the duplicate values comprises:

[0029] deleting the duplicate values and the discrete missing data;

[0030] filling the continuous missing data with the mean value.

[0031] Optionally, the abnormality detection model based on the Gaussian probability distribution comprises:

[0032] data preparation: taking the final extracted feature vectors and the constructed feature vectors as inputs;

[0033] establishing the abnormality detection model based on the Gaussian probability distribution;

[0034] setting an abnormality probability threshold and calculating a model evaluation index;

[0035] feedback iteration and optimization: repeatedly testing the performance of the abnormality detection model under different parameters, verifying the effectiveness of the feature selection and extraction and construction by the difference in the performance of the abnormality detection model on different feature sets, and outputting the feature vectors and the corresponding evaluation indexes.

[0036] According to a second aspect of the present application, a feature screening system for civil aviation price abnormality detection is provided, and the feature screening system comprises:

[0037] a data cleaning module capable of cleaning data of an original price calculation log file;

[0038] a feature selection and extraction module capable of screening all effective information having a significant influence on the civil aviation price abnormality detection task from the cleaned price calculation log file, constructing an effective information set, and extracting feature vectors;

[0039] a feature construction module capable of constructing feature vectors based on construction rules provided by business experts and business features affecting civil aviation prices in civil aviation business scenarios;

[0040] a feature analysis module capable of verifying the effectiveness of the extracted feature vectors and the constructed feature vectors.

[0041] According to a third aspect of the present application, an electronic device is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above method when executing the program.

[0042] Beneficial technical effects:

[0043] 1. According to the complex characteristics of the civil aviation price log data, the application uses advanced machine learning algorithms and data mining methods to systematically propose a feature screening method and system for civil aviation price anomaly detection from the aspects of data cleaning, feature selection and extraction, feature construction, and feature analysis, aiming to systematically process civil aviation price data and accurately and efficiently screen out key features affecting civil aviation prices, providing data feature vectors with high-value semantic information for civil aviation anomaly detection tasks, and improving the accuracy and reliability of subsequent algorithm analysis.

[0044] 2. The application can reduce the dimensionality of the feature vector. The feature vector screened by the application can be used as the input of the subsequent abnormal price detection algorithm to achieve high-precision identification of abnormal prices, and is beneficial to reduce the computing power and storage cost of the detection system.

[0045] Other features and advantages of the application will be set forth in the following description, and in part will become apparent to those skilled in the art, and will be learned from the description, or will be learned by practicing the application. The purpose and other advantages of the application can be achieved and obtained by the structure indicated in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0047] Figure 1 A step flowchart of the feature screening method for civil aviation price anomaly detection according to an embodiment of the application is shown.

[0048] Figure 2 A step flowchart of data cleaning in an embodiment of the application is shown.

[0049] Figure 3 The source of the original price calculation log file in an embodiment of the application is shown.

[0050] Figure 4 A step flowchart of constructing a feature vector in an embodiment of the application is shown.

[0051] Figure 5 A step flowchart of feature analysis in an embodiment of the application is shown.

[0052] Figure 6 A step flowchart of validity verification in an embodiment of the application is shown.

[0053] Figure 7 An example architecture diagram of a feature screening system for detecting anomalies in civil aviation fares according to an embodiment of this application is shown.

[0054] Figure 8 An example architectural diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0057] Figure 1 A flowchart illustrating the steps of a feature selection method for detecting anomalies in civil aviation fares according to an embodiment of this application is shown. Figure 1 As shown, the feature screening system may include the following steps S1 to S4.

[0058] S1: Clean the original freight rate calculation log file. This step is fundamental to the entire feature selection process.

[0059] Among them, such as Figure 2 As shown, the data cleaning of the original freight rate calculation log file may include the following steps S11 to S13.

[0060] S11: Read the original fare calculation log file, filter and organize it, and retain the fields in the original fare calculation log file that are of practical significance for civil aviation fare calculation.

[0061] It should be noted that the freight rate data in this application comes from the original freight rate calculation log file. For example... Figure 3As shown, the computing engine of the civil aviation pricing system will perform real-time pricing calculation according to the pricing rules and related data in the pricing database after receiving an external pricing request, and will generate transaction-related log records; by analyzing and pushing the historical or real-time transaction logs, the pricing data can be obtained. Therefore, the source of the pricing data of the present application is the original pricing calculation log file, i.e. the original pricing calculation log file.

[0062] In some embodiments of the present application, the original pricing calculation log file can be read in an offline or online manner. These log files contain a large number of transaction records, each record including multiple fields such as ticket price, departure location, destination, passenger type, airline, etc.

[0063] After reading the original pricing calculation log file, screening and sorting are performed by pattern matching, and fields such as system identifiers and prompt information that have no actual significance for civil aviation pricing calculation are deleted, and fields such as flight origin, start and end time, and ticket price that have actual significance for civil aviation pricing calculation are retained.

[0064] S12: Check each field, identify and process missing values and duplicate values to ensure data accuracy and integrity.

[0065] In some embodiments of the present application, the missing values include continuous missing data and discrete missing data; wherein the continuous missing data such as ticket price and departure time, and the discrete missing data such as flight cabin section and route origin. The identification and processing of missing values and duplicate values include: deleting duplicate values and discrete missing data; using mean value to fill continuous missing data. By deleting duplicate values, the uniqueness and accuracy of the data can be ensured. For discrete missing data, due to its criticality and particularity, system abnormal output is implemented, and this piece of pricing data is screened out, which can avoid adverse effects on the overall quality.

[0066] S13: Frequency encoding of discrete data, converting discrete data into continuous numerical data.

[0067] In some embodiments of the present application, the discrete data after cleaning is uniformly encoded, frequency encoding is used, and the frequency information of the discrete data is retained for subsequent data analysis and feature extraction. The frequency encoding method is as follows:

[0068] First, let the input data set be D, which contains different categories of discrete variables C.

[0069] Second, calculate the number of categories and the total number of samples: for each category c i ∈C, calculate the number of times it appears in the data set, denoted as count(c i). Let the total sample number be N, i.e. N = |D|.

[0070] Then calculate the frequency: for each category c i , calculate the frequency

[0071] Finally replace the original category value: replace each category c i in the data set with its corresponding frequency value f(c i ).

[0072] S2: Filter all valid information that has a significant impact on the civil aviation price anomaly detection task from the cleaned airfare calculation log file to form a valid information set, and extract a feature vector.

[0073] In some embodiments of the present application, the valid information includes ticket price, takeoff time, landing time, booking time, passenger type, origin and destination, etc. In addition, flight distance, flight duration, and holiday information with semantic information should also be considered.

[0074] In some embodiments of the present application, the step S2 uses principal component analysis (PCA) to extract the feature vector from the valid information set to improve the performance of the final airfare anomaly detection model. Specifically, the valid information in each cleaned log file is spliced into an information vector, and the vectors generated by multiple log files are spliced into an information matrix, and linear transformation is performed to convert to a new space, so that the variance of the data in each dimension in the new space is maximized, and each of the extracted feature vectors can represent the distribution of the original data to the greatest extent. The specific steps are as follows:

[0075] First, generate an information matrix: assuming that there are m elements in the valid information set, for each log file, generate an m-dimensional information vector (row vector), and splice the information vectors of n log files to generate an information matrix X n×m , where n should be much larger than m to improve the representativeness of the extracted features.

[0076] Second, standardize the information matrix. The conversion formula is:

[0077]

[0078] Where μ j and σ j are the mean and standard deviation of the jth feature, respectively, and Z is the standardized data.

[0079] Third, calculate the sample covariance matrix C. The calculation formula is:

[0080]

[0081] Fourth, eigenvalue decomposition: Calculate the eigenvalues ​​and eigenvectors of the covariance matrix C; the formula is as follows:

[0082] Cv i =λ i v i i = 1, ..., m

[0083] Where, λ i For the i-th eigenvalue, v i It is its corresponding feature vector.

[0084] Fifth, select principal components: Sort the eigenvalues ​​in descending order and select the first k eigenvectors v1, ..., v2. k As a principal component, k should be less than m.

[0085] Sixth, linear data transformation: linearly projecting the original data onto a new principal component space. The transformation formula is as follows:

[0086] Y = ZV k

[0087] Among them, V k = [v1, ..., v k Y is the new feature matrix after transformation, where each i-th row represents the feature vector corresponding to the i-th log file.

[0088] S3: Construct feature vectors based on the construction rules provided by business experts and the business characteristics that affect civil aviation fares in civil aviation business scenarios.

[0089] Among them, such as Figure 4 As shown, the feature vector constructed based on the construction rules provided by business experts and the business characteristics affecting airfares in civil aviation business scenarios includes:

[0090] S31: Perform business scenario modeling, identify the business characteristics affecting civil aviation fares from the cleaned fare calculation log files, and construct a business characteristic set. These business characteristics include departure time, landing time, booking time, destination, etc.

[0091] S32: Construct features using business features from the business feature set, based on the construction rules provided by business experts.

[0092] In one embodiment of this application, the business expert provides the following construction rule: Red-eye flights generally refer to flights with a departure time before 7:00 AM and a landing time after 10:00 PM. Based on this construction rule, the following features are constructed using the departure and arrival times from the business feature set:

[0093] F Red-Eye =I(T) departure <07:00)×I(Tarrival >22:00)

[0094] wherein, I (condition) is an indicator function, I (condition) = 1 when the condition (i.e. condition) is true; otherwise, I (condition) = 0. Red-Eye denotes the red-eye flight feature, T departure and T arrival are the departure time and the arrival time, respectively.

[0095] In an embodiment of the present application, the following features are constructed based on the construction rules and business features (flight distance and flight time) provided by business experts:

[0096]

[0097] wherein, D is the flight distance (in kilometers), T is the flight time (in hours), F stopover is the stopover flight feature.

[0098] In some embodiments of the present application, features such as the identification feature, the passenger identity feature and the cabin opening situation feature cannot be sold separately, which can be determined by identifying special fields in the log and using regular matching.

[0099] S33: Combine the constructed features into a feature vector.

[0100] In step S3 of the present application, for the civil aviation fare anomaly detection scenario, the business features that have a greater impact on the fare in the civil aviation business scenario are analyzed, and new features are constructed based on the original feature dimension rule matching method. For example, starting from the basic features such as the takeoff time, the landing time and the booking time of the aircraft, more semantic information and more direct features associated with ticket pricing such as red-eye flights, off-peak seasons, holidays and the like can be further extracted. By constructing a feature vector, the feature set can be enriched and expanded, the data expression capability can be enhanced, and the upper limit of the anomaly detection model can be further improved.

[0101] S4: Verify the effectiveness of the extracted feature vector and the constructed feature vector to ensure that they have a significant impact on the civil aviation fare. As shown in Figure 5 , the effectiveness verification of the extracted feature vector and the constructed feature vector includes:

[0102] S41: Analyze the explanatory power of the extracted feature vector, and take the feature variable with a cumulative variance contribution rate of 90% as the final extracted feature vector.

[0103] The feature extracted by the principal component analysis (PCA) is analyzed for feature interpretation. The eigenvalue of the covariance matrix can be calculated during the principal component analysis, and the eigenvalue reflects the size of the variance that can be explained by each principal component. Therefore, the eigenvalue size can be used to judge the extraction effect of the principal component analysis, and the number of principal components is set accordingly.

[0104] Specifically, during the interpretation analysis, the eigenvalue corresponding to each principal component is taken as a reference index of the interpretation, and the relationship between the number of principal components and the overall interpretation is checked to guide the determination of the number of final principal components. Each eigenvalue in the principal component analysis corresponds to a principal component, and reflects the interpretation degree of the principal component to the variance of the original data. The larger the eigenvalue, the more data variance that can be explained by the principal component, and the stronger the interpretation.

[0105] Therefore, the variance contribution rate of each principal component is:

[0106]

[0107] where k is the number of principal components, λ i is the ith eigenvalue.

[0108] The eigenvalues of all principal components are sorted, and a relationship diagram of the eigenvalues and the number of principal components (i.e., a cumulative variance contribution rate diagram) is drawn. By observing the relationship diagram, we can identify which principal components have strong interpretation and at which point the newly added principal components have smaller interpretation contribution to the data. The number of principal components when the cumulative variance contribution rate reaches 90% is used in the present application.

[0109] In step S41 of the present application, the interpretation analysis of the extracted feature vector can simplify the data dimension while retaining most of the information, thereby improving the calculation efficiency while ensuring the analysis accuracy.

[0110] S42: The final extracted feature vector and the constructed feature vector are verified for effectiveness by the Gaussian probability distribution-based anomaly detection model. As shown in Figure 6 The Gaussian probability distribution-based anomaly detection model for verifying the effectiveness of the final extracted feature vector and the constructed feature vector includes:

[0111] S421: Data preparation: taking the final extracted feature vector and the constructed feature vector as input.

[0112] S422: Establishing a Gaussian probability distribution-based anomaly detection model. Specifically, the sample data is first processed

[0113] where μ and σ are the sample mean and standard deviation in each category, respectively.

[0114] S423: Set the anomaly probability threshold, and calculate the evaluation index of the model. The evaluation index can include accuracy, recall rate, and F1 score, etc.

[0115] S424: Feedback iteration and optimization: repeatedly test the performance of the anomaly detection model under different parameters, use the difference in the performance of the anomaly detection model on different feature sets to verify the effectiveness of feature selection and construction, and output the feature vector and the corresponding evaluation index.

[0116] In step S42 of the present application, the anomaly detection model based on Gaussian probability distribution fits the normal data into a Gaussian normal distribution, and the data outside the probability threshold is regarded as an anomaly value. The algorithm of the model is simple and can reflect the effectiveness of data feature selection and extraction and construction.

[0117] Based on the above feature screening method, the present application provides a feature screening system for civil aviation price anomaly detection, which is described below.

[0118] The feature screening system for civil aviation price anomaly detection provided by the embodiments of the present application is described below, and the feature screening system described below can be correspondingly referred to the feature screening method described above.

[0119] Figure 7 An architecture example diagram of the feature screening system for civil aviation price anomaly detection according to the embodiments of the present application is shown. As shown in the figure, the feature screening system includes a data cleaning module 20, a feature selection and extraction module 30, a feature construction module 40, and a feature analysis module 50. Figure 7 The data cleaning module 20 is used for data cleaning of the original price calculation log file. The data cleaning of the original price calculation log file includes reading the original price calculation log file, screening and sorting, and retaining the fields in the original price calculation log file that have actual significance for civil aviation price calculation; checking each field, identifying and processing missing values and duplicate values; and frequency encoding discrete data to convert discrete data into continuous numerical data. The input of the data cleaning module 20 is the original price calculation log file, and the output is the cleaned log file.

[0120] The feature selection and extraction module 30 is used for screening all effective information that has a significant impact on the civil aviation price anomaly detection task from the cleaned price calculation log file, constituting an effective information set, and extracting a feature vector. The input of the feature selection and extraction module 30 is the cleaned log file, and the output is the extracted feature vector.

[0121]

[0122] ​The feature construction module 40 is configured to construct a feature vector based on a construction rule provided by a business expert and a business feature in a civil aviation business scenario that affects the civil aviation freight rate; wherein the construction of the feature vector based on the construction rule provided by the business expert and the business feature in the civil aviation business scenario that affects the civil aviation freight rate comprises: performing business scenario modeling, determining the business features that affect the civil aviation freight rate from the cleaned freight rate calculation log file to form a business feature set; constructing a feature using the business features in the business feature set according to the construction rule provided by the business expert; and combining the constructed features into a feature vector. The input of the feature construction module 40 is the cleaned log file, and the output is the constructed feature vector.

[0123] The feature analysis module 50 is configured to verify the effectiveness of the extracted feature vector and the constructed feature vector. Wherein the effectiveness verification of the extracted feature vector and the constructed feature vector comprises: performing an explanation power analysis on the extracted feature vector, and taking a feature variable with a cumulative variance contribution rate of 90% as a final extracted feature vector; and performing effectiveness verification on the final extracted feature vector and the constructed feature vector based on a Gaussian probability distribution-based anomaly detection model. The input of the feature analysis module 50 is the extracted feature vector and the constructed feature vector, and the output is the feature vector and an evaluation index.

[0124] The feature screening method and system for civil aviation freight rate anomaly detection of the present application have the following beneficial effects:

[0125] 1. The present application is aimed at the complex form of civil aviation freight rate log data, and uses advanced machine learning algorithms and data mining methods to systematically propose a feature screening method and system for civil aviation freight rate anomaly detection from the aspects of data cleaning, feature selection and extraction, feature construction, and feature analysis, which aims to systematically process civil aviation freight rate data and accurately and efficiently screen out key features that affect civil aviation freight rates, providing data feature vectors with high-value semantic information for civil aviation anomaly detection tasks, and improving the accuracy and reliability of subsequent algorithm analysis.

[0126] 2. The present application can reduce the dimensionality of the feature vector, and the feature vector screened by the present application can be used as the input of the subsequent anomaly freight rate detection algorithm, which can realize high-precision identification of abnormal prices, and is beneficial to reducing the computing power and storage cost of the detection system.

[0127] In addition, the feature screening method and system for civil aviation freight rate anomaly detection of the present application solve the technical problem that there is no good feature screening method that can accurately and efficiently extract data features with a larger pricing correlation factor from civil aviation freight rate data, and lay a foundation for the use of subsequent anomaly detection models in the field of civil aviation freight rates.

[0128] Figure 8An example architectural diagram of an electronic device according to an embodiment of this application is shown. Figure 8 As shown, the electronic device includes a processor 110, a communication interface 120, a memory 130 for storing processor-executable instructions, and a communication bus 140. The processor 110, communication interface 120, and memory 130 communicate with each other via the communication bus 140. The processor 50 implements the aforementioned feature selection method by running an executable computer program.

[0129] The computer program in memory 130, when implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0130] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected based on actual needs to achieve the purpose of this embodiment. Those skilled in the art can understand and implement this without any creative effort.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0132] Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood that modifications can be made to the foregoing embodiments, or additional implementations can be implemented, without departing from the spirit and scope of the embodiments disclosed in the present application.

Claims

1. A feature screening method for civil aviation fare anomaly detection, characterized in that, The method comprises: data cleaning on the original freight calculation log file; screening all valid information from the cleaned freight calculation log file which has a significant impact on the civil aviation freight abnormality detection task, constituting an effective information set, and extracting a feature vector; constructing a feature vector based on the construction rules provided by business experts and the business features affecting civil aviation freight in the civil aviation business scenario; validity verification of the extracted feature vector and the constructed feature vector; the validity verification of the extracted feature vector and the constructed feature vector comprises: performing explanatory power analysis on the extracted feature vector, and taking the feature variables with a cumulative variance contribution rate of 90% as the final extracted feature vector; performing validity verification on the final extracted feature vector and the constructed feature vector based on a Gaussian probability distribution-based abnormality detection model; the validity verification on the final extracted feature vector and the constructed feature vector based on the Gaussian probability distribution-based abnormality detection model comprises: data preparation: taking the final extracted feature vector and the constructed feature vector as input; establishing a Gaussian probability distribution-based abnormality detection model; setting an abnormal probability threshold, calculating model evaluation indicators; feedback iteration and optimization: repeatedly testing the performance of the abnormality detection model under different parameters, verifying the effectiveness of feature selection and construction by the difference in the performance of the abnormality detection model on different feature sets, and outputting the feature vector and the corresponding evaluation indicators.

2. The feature screening method for civil aviation price anomaly detection according to claim 1, characterized in that, the data cleaning on the original freight calculation log file comprises: reading the original freight calculation log file, screening and organizing, and retaining the fields in the original freight calculation log file which have actual significance for civil aviation freight calculation; checking each field, identifying and processing missing values and duplicate values; frequency encoding is performed on discrete data to convert discrete data into continuous numerical data.

3. The feature screening method for civil aviation price anomaly detection according to claim 1, characterized in that, the construction of the feature vector based on the construction rules provided by business experts and the business features affecting civil aviation freight in the civil aviation business scenario comprises: performing business scenario modeling to determine the business features affecting civil aviation freight from the cleaned freight calculation log file, constituting a business feature set; constructing features using the business features in the business feature set according to the construction rules provided by business experts; combining the constructed features into a feature vector.

4. The feature screening method for civil aviation price anomaly detection according to claim 1, characterized in that, all valid information which has a significant impact on the civil aviation freight abnormality detection task includes ticket price, takeoff time, landing time, booking time, passenger type, origin and destination.

5. The feature screening method for civil aviation price anomaly detection according to claim 1, characterized in that, principal component analysis is used to extract a feature vector from the effective information set.

6. The feature screening method for civil aviation price anomaly detection according to claim 2, characterized in that, the missing values include continuous missing data and discrete missing data; the identification and processing of missing values and duplicate values comprise: deleting duplicate values and discrete missing data; using mean value to fill continuous missing data.

7. A feature selection system for civil aviation fare anomaly detection, applying the method of any one of claims 1-6, characterized in that, comprise: a data cleaning module capable of cleaning the original freight calculation log file; a feature selection and extraction module capable of screening all valid information from the cleaned freight calculation log file which has a significant impact on the civil aviation freight abnormality detection task, constituting an effective information set, and extracting a feature vector; The feature construction module is capable of constructing a feature vector based on a construction rule provided by a business expert and a business feature in a civil aviation business scenario that influences a civil aviation fare; The feature analysis module is capable of performing feature effectiveness verification on the extracted feature vector and the constructed feature vector.

8. An electronic device, comprising: The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the steps of the method of any one of claims 1-6 when executing the program.

Images