A method for extracting key features of large-scale data, electronic equipment and medium
Through the adaptive density threshold and density clustering algorithm, the problem of outliers in pharmaceutical sales data is solved, the accuracy of stability feature extraction of drug sales data is improved, and the authenticity of the data is ensured.
Patent Information
- Application Number
- CN202411261799.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-10
AI Technical Summary
In the existing medical sales data analysis, the existence of outliers leads to deviations in the stability analysis of drug sales data. Traditional clustering algorithms are prone to missed or missed screening threshold settings, reducing the accuracy of drug sales data.
The adaptive density threshold calculation method is used to obtain the fluctuation of drug sales and the co-occurrence probability of related drugs, calculate the synchronous change difference weight and correlation deviation value, and eliminate outliers in combination with the density clustering algorithm to obtain the stable characteristic sequence of the drug.
It improves the accuracy of the stability feature extraction of drug sales data, reduces interference from outliers, and ensures the true stability analysis of drug sales data.
Smart Images

Figure CN118797315B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of big data analysis and processing, and specifically to a method for extracting key features of large-scale data, an electronic device and a medium. Background Art
[0002] In the pharmaceutical field, the big data platform is used to analyze the characteristics of pharmaceutical data. The analysis of pharmaceutical sales data can help us understand market dynamics, optimize inventory management, formulate sales strategies, and improve drug research and development. However, the pharmaceutical sales in a single region are diverse, and the sources of drug sales channels are diverse. Due to the randomness of some people purchasing drugs, sales data will fluctuate to a certain extent. At the same time, due to incomplete data collection, record errors, and missing data, drug sales data will have abnormal values.
[0003] The existence of outlier sales values is not conducive to the analysis of the stability of drug sales data, and it is easy to cause deviations in the characteristics of drug stability. At present, clustering algorithms are usually used to exclude outliers in pharmaceutical sales data. When clustering sales data, a fixed screening threshold is often set; however, for different types of drugs, their sales volume and sales fluctuations are different, so it is easy to miss outliers or misjudge normal fluctuations as outliers, reducing the accuracy of drug sales data stability feature extraction. Summary of the invention
[0004] In order to solve the above technical problems, the present application provides a large-scale data key feature extraction method to solve the existing problems.
[0005] A large-scale data key feature extraction method of the present application adopts the following technical solution:
[0006] In a first aspect, an embodiment of the present application provides a method for extracting key features of large-scale data, the method comprising the following steps:
[0007] Collect the daily sales volume of each type of medicine in each channel in the target month;
[0008] According to the numerical distribution of daily sales of various drugs in all channels, the fluctuation degree of drug sales of various drugs is obtained; all prescriptions of the target month are obtained; according to the co-occurrence probability of various drugs with other drugs in the prescription, the related drugs of various drugs are obtained;
[0009] According to the difference in the daily sales volume of each drug type and its associated drugs and the difference in the degree of fluctuation in drug sales, the weight of the synchronous change difference between each drug type and its associated drugs is calculated; according to the distance between the daily sales series of each drug type and its associated drugs and the degree of fluctuation in drug sales, the associated deviation value of each drug type and its associated drugs is obtained; according to the weight of the synchronous change difference and the associated deviation value of each drug type and its all associated drugs, the data fluctuation abnormal value is obtained;
[0010] An adaptive density threshold is obtained according to the abnormal value of data fluctuation of each type of drug; the adaptive density threshold is substituted into the density clustering algorithm to cluster all daily sales of each type of drug, and a stable characteristic sequence of drugs in the target month is obtained.
[0011] Furthermore, the fluctuation degree of drug sales volume of each drug is obtained according to the numerical distribution of daily sales volume of each drug in all channels, including:
[0012] The sum of the daily sales volume of each type of drug in all channels is recorded as the daily sales volume of each type of drug; the sequence composed of all daily sales volume of each type of drug in the target month is recorded as the daily sales sequence of each type of drug; the daily sales sequence of each type of drug is fitted with the least square method to obtain the sales curve; and the extreme value points on the sales curve are obtained;
[0013] Calculate the difference between the daily sales volume corresponding to each extreme point in the daily sales sequence and its previous daily sales volume, and record it as the first difference; calculate the difference between the daily sales volume corresponding to each extreme point in the daily sales sequence and its subsequent daily sales volume, and record it as the second difference; calculate the sum of the first difference and the second difference; and add up the sum of all extreme points on the sales curve, and record it as the degree of fluctuation of drug sales volume of each type of drug.
[0014] Furthermore, according to the co-occurrence probability of each drug and other drugs in the prescription, the associated drugs of each drug are obtained, including:
[0015] Any type of drug is recorded as the target drug, and the co-occurrence probability of the target drug and other types of drugs except the target drug in all prescriptions of the target month is calculated; the preset number of types of drugs with the highest co-occurrence probability in all prescriptions of the target month are recorded as the associated drugs of the target drug.
[0016] Furthermore, the method for calculating the synchronous change difference weight between each class of drugs and each of its associated drugs includes: recording the synchronous change difference weight between the b-th class of drugs and its c-th associated drug as , ; In the formula, Daily sales sequence for category b drugs The variance of The daily sales sequence of the cth associated drug of the bth drug The variance of The degree of fluctuation in drug sales for Category B drugs; The degree of fluctuation in drug sales of the cth associated drug of the bth class drug; is a linear normalization function.
[0017] Furthermore, the method for obtaining the associated deviation value of each associated drug of each type of drug includes:
[0018] For each type of drug, the distance between the daily sales series of the drug and its associated drugs is calculated; the product of the distance and the degree of fluctuation of the drug sales of each associated drug of each type of drug is used as the association deviation value between the drug and its associated drugs.
[0019] Furthermore, the data fluctuation abnormal value is obtained according to the synchronous change difference weight and associated deviation value of each type of drug and all its associated drugs, including:
[0020] The fusion results of the synchronous change difference weights and associated deviation values of each type of drug and its associated drugs are calculated, and the average of the fusion results of each type of drug and all its associated drugs is used as the data fluctuation abnormal value of each type of drug.
[0021] Further, the adaptive density threshold is obtained according to the abnormal value of the data fluctuation of each type of medicine, including: recording the adaptive density threshold corresponding to the b-th type of medicine as , ;in, is the adaptive density threshold corresponding to the b-th drug; is the mean of the daily sales series of class b drugs; The data fluctuation abnormal value is for the Class B drugs; is the preset initial density threshold; is a linear normalization function; is the rounding function; It is the preset control parameter.
[0022] Furthermore, the adaptive density threshold is substituted into the density clustering algorithm to cluster all daily sales of each type of drug, and a stable feature sequence of drugs in the target month is obtained, including:
[0023] The adaptive density threshold is used as the density threshold in the density clustering algorithm to cluster the daily sales sequence of each type of drug to obtain each cluster; the cluster containing the most elements in all clusters is marked as a normal cluster;
[0024] The ratio of the number of elements contained in the normal cluster to the number of elements in the daily sales sequence of the drug class is recorded as the normal rate of sales data of each drug class; the product of the variance of all elements in the normal cluster and the normal rate of sales data of each drug class is used as the stability judgment value of each drug class;
[0025] All drugs are sorted according to their stability evaluation values to obtain the drug stability characteristic sequence for the target month.
[0026] In a second aspect, an embodiment of the present application provides an electronic device for extracting key features of large-scale data, and the electronic device for extracting key features of large-scale data includes:
[0027] Big data collection module, used to collect daily sales of various types of drugs in each channel in the target month;
[0028] The big data analysis module is used to obtain the degree of fluctuation of drug sales of each type of drug according to the numerical distribution of daily sales of each type of drug in all channels; obtain all prescriptions of the target month; obtain the related drugs of each type of drug according to the co-occurrence probability of each type of drug and other types of drugs in the prescription; calculate the weight of the synchronous change difference between each type of drug and its related drugs according to the difference in the change of all daily sales of each type of drug and its related drugs and the difference in the degree of drug sales fluctuation; obtain the associated deviation value of each type of drug and its related drugs according to the distance between the daily sales series of each type of drug and its related drugs and the degree of drug sales fluctuation; obtain the data fluctuation abnormal value according to the weight of the synchronous change difference and the associated deviation value of each type of drug and all its related drugs;
[0029] The data feature extraction module is used to obtain an adaptive density threshold according to the abnormal data fluctuation values of each type of drug; the adaptive density threshold is substituted into the density clustering algorithm to cluster all daily sales of each type of drug to obtain a stable feature sequence of the drug in the target month.
[0030] In a third aspect, an embodiment of the present application further provides a large-scale data key feature extraction medium, wherein the data key feature extraction medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned large-scale data key feature extraction methods are implemented.
[0031] This application has at least the following beneficial effects:
[0032] This application obtains the overall sales data of various types of drugs in a certain area through a time monitoring tool and uploads it to a big data platform for analysis. According to the time-dependent characteristics of daily sales in the daily sales sequence of a single type of drug, the degree of fluctuation in drug sales is obtained, which reflects the degree to which the drug is subject to normal fluctuations of randomness and seasonality, and the data fluctuation abnormal values are obtained by using the similarity of fluctuation changes in sales data of drugs associated with this type of drug, which reflects the correlation of drug collaborative transformation, thereby obtaining the adaptive density threshold of a single type of drug in the target month, and using the clustering algorithm to obtain normal clusters and abnormal clusters, eliminating the interference of abnormal values to obtain the true stability of drug sales. It solves the risk of large deviations between stability and actual sales when all sales are directly used for stability analysis in traditional algorithms, resulting in the defect of inaccurate stability characteristics. In this solution, the adaptive density threshold of the clustering algorithm is controlled for the characteristics of time-dependent characteristics and similarity changes of associated drugs, and the screening accuracy of abnormal values is improved, so that more accurate daily sales data can be obtained, thereby improving the accuracy of extracting drug sales stability features. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0034] Figure 1 A flowchart of a method for extracting key features of large-scale data provided by an embodiment of the present application;
[0035] Figure 2 A block diagram for obtaining a data fluctuation difference value provided by an embodiment of the present application;
[0036] Figure 3 A schematic diagram of the structure of an electronic device for extracting key features of large-scale data is provided as an embodiment of the present application. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0038] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.
[0039] The specific scheme of a large-scale data key feature extraction method provided by the present application is described in detail below with reference to the accompanying drawings.
[0040] An embodiment of the present application provides a method for extracting key features of large-scale data. Specifically, the following method for extracting key features of large-scale data is provided. Figure 1 , the method comprises the following steps:
[0041] Step S1, collecting the daily sales volume of each type of medicine in each channel in the target month.
[0042] A specific implementation scenario of the embodiment of the present application is the stability feature extraction scenario of drug sales data. When using the big data platform to perform feature analysis on medical data, the source of drug sales channels and data loss during the collection process may cause errors in the stability analysis of drug sales data. Therefore, it is necessary to record the sales data of various drugs under various channels on a daily basis and analyze the stability of drug sales data.
[0043] Specifically, this embodiment takes City A as an example and uses the MAX medical omni-channel market analysis tool to obtain the daily sales volume of various types of drugs in each channel in City A in the target month on a daily basis, wherein the channels include the core hospital market, community health center market, county hospital market, township health center market, DTP pharmacy market and B2C electric field market.
[0044] Due to the wide variety of drugs, according to the National Drug Catalog 2018 Edition, chemical drugs and biological products are classified according to clinical pharmacology, with a total of 417 varieties, and Chinese patent medicines are classified according to function, with a total of 268 varieties. The amount of data summarized from all categories of drug data under various channels in City A is relatively large, so the MAX monitoring tool is needed to obtain the sales data of various types of drugs, upload it to the HadooP big data processing platform, and perform data analysis on the big data processing platform to analyze the fluctuations of continuous daily sales data of drugs, eliminate the impact of abnormal data caused by data loss on stability analysis, and extract the stability characteristics of drug sales.
[0045] Step S2, according to the difference in the numerical distribution of the daily sales volume of each type of medicine and other medicines in all channels, obtain the data fluctuation abnormal value of each type of medicine.
[0046] In daily sales of a single type of drug, there may be a certain degree of fluctuation due to random purchases by patients. However, for the entire region, due to the large population base, the total number of patients is relatively fixed, so the amplitude of random fluctuations in daily sales of a single type of drug is small. Due to weather or seasonal factors, the fluctuations in daily sales data of a single type of drug have the characteristics of front-end dependence. However, if the abnormal sales data caused by factors such as statistical errors are hidden in the random fluctuations of daily sales and the fluctuations of external influences, it will interfere with the feature extraction of the stability of drug sales data, which requires further analysis.
[0047] S21, based on the numerical distribution of daily sales of various drugs in all channels, obtain the degree of fluctuation of drug sales of various drugs.
[0048] In the daily sales series of a single type of drug, random purchases by the patient group or the influence of environmental factors cause small fluctuations in drug sales, and the sales volume has a time-dependent characteristic; however, if the abnormal sales value is caused by data accuracy factors such as statistical errors, the time dependence of the entire series is relatively weak.
[0049] The sum of daily sales of various types of drugs in all channels is recorded as the daily sales volume of various types of drugs; the sequence composed of all daily sales of various types of drugs in the target month is recorded as the daily sales sequence of various types of drugs; the daily sales sequence of various types of drugs is curve fitted by the least squares method to obtain the sales curve; the extreme points on the sales curve are obtained; the difference between the daily sales volume corresponding to each extreme point in the daily sales sequence and the sales volume of the previous day is calculated, and recorded as the first difference; the difference between the daily sales volume corresponding to each extreme point in the daily sales sequence and the sales volume of the next day is calculated, and recorded as the second difference; the sum of the first difference and the second difference is calculated, and recorded as the sales volume difference amplitude corresponding to each extreme point; the cumulative sum of the sum values of all extreme points on the sales curve is recorded as the degree of fluctuation of drug sales volume of various types of drugs.
[0050] It should be noted that the difference represents the numerical difference between two data, and in this embodiment, the absolute value of the difference is calculated.
[0051] If the sales data of this type of drug in the target month changes only due to random fluctuations in the patient population or environmental factors, the overall fluctuation of the drug sales data is small, and the deviation of the sequence extreme point in the local range is small, that is, the total deviation of a single extreme point is small. At the same time, if an epidemic disease caused by environmental changes occurs in this month, causing the drug sales data to fluctuate, the epidemic disease has a certain time dependence due to environmental changes, that is, the daily sales of drugs gradually change over time, so that the total deviation of the extreme point on the sales curve is relatively small, and the value of the drug sales fluctuation degree of this type of drug in the target month is small. On the contrary, if the abnormal daily sales data caused by statistical reasons has a large deviation within the range, that is, the value of the total deviation corresponding to the extreme point is large, and the final value of the drug sales fluctuation degree is large.
[0052] S22, obtain all prescriptions for the target month; obtain the related drugs of each drug category based on the co-occurrence probability of each drug category and other drugs in the prescription; calculate the weight of the synchronous change difference between each drug category and its related drugs based on the difference in the change of all daily sales volume of each drug category and its related drugs and the difference in the degree of fluctuation of drug sales volume.
[0053] By analyzing the daily sales sequence of a single type of drug in a single month, we can obtain the corresponding degree of fluctuation in drug sales, reflecting the degree of abnormality in the drug sales data. However, the deviations of some statistical errors are relatively small, which is not conducive to distinguishing them from normal fluctuations. Further analysis of the drug sales data is needed.
[0054] Since the human body is a whole, when a patient has a certain disease, other complications often occur. Therefore, when treating patients, it is necessary to consider the patient's condition as a whole, and when treating patients with drugs, multiple drugs are often needed to treat them together. For example, patients with coronary heart disease often have high blood pressure, and they need to use antihypertensive drugs while using anticoagulants; for osteoporosis patients, they need to use vitamin D while supplementing calcium to promote absorption. Therefore, there is a promotion and synergistic relationship between drugs. When analyzing the sales data of a certain type of drug, it is necessary to analyze the related drugs to determine whether there is any abnormality in the sales volume of this type of drug.
[0055] First, obtain N types of drugs associated with each type of drug. The acquisition method is to obtain all prescriptions for the target month; record any type of drug as the target drug, calculate the co-occurrence probability of the target drug and other types of drugs except the target drug in all prescriptions for the target month; and take the preset number N=5 types of drugs with the highest co-occurrence probability in all prescriptions for the target month as the associated drugs of the target drug.
[0056] It should be noted that the co-occurrence probability is a well-known technology in the field of natural language processing, which indicates the probability of a target drug appearing in the prescriptions of other types of drugs. It can be obtained by constructing a co-occurrence matrix between all drugs, wherein the element in the i-th row and j-th column of the co-occurrence matrix indicates the number of times drug j appears in the prescription where drug i is located. The sum of the number of times drug j and drug i appear in the prescription is calculated and recorded as the cumulative number of appearances. The co-occurrence probability of drug j and drug i is the ratio of the number of times drug j appears in the prescription where drug i is located to the cumulative number of appearances.
[0057] Taking the b-type drug in the target month sales data as an example, the synchronous change difference weight is calculated based on the fluctuation difference between the daily sales series of b-type drugs and the associated N types of drugs: ; In the formula, is the weight of the synchronous change difference between the b-th drug and its c-th associated drug; Daily sales sequence for category b drugs The variance of The daily sales sequence of the cth associated drug of the bth drug The variance of The degree of fluctuation in drug sales for Category B drugs; The degree of fluctuation in drug sales of the cth associated drug of the bth class drug; is a linear normalization function.
[0058] If only random and seasonal fluctuations exist in Category B, then the fluctuations of Category B and Category C drugs in the daily sales sequence are similar. Therefore, the difference between the daily sales sequence and the degree of fluctuation of drug sales of the two types of drugs is small. Therefore, when the difference in the variance of the daily sales sequence of the two types of drugs is small, and the difference in the degree of fluctuation of drug sales calculated for the two types of drugs is small, the final weight of the synchronous change difference is small, indicating that the possibility of abnormal sales of this type of drug is small.
[0059] S23, obtaining the correlation deviation value of each drug and its associated drugs according to the distance between the daily sales series of each drug and the degree of fluctuation of drug sales.
[0060] Calculate the distance between the daily sales series of the b-th drug and its c-th associated drug , where the optional distance calculation methods are DTW distance, Mahalanobis distance and Euclidean distance, all of which are well-known technologies. The implementer can choose according to the actual situation, and this application does not limit it. Calculate the association deviation value between the b-th drug and its c-th associated drug , ;in, is the association deviation value between the b-th drug and its c-th associated drug; The degree of fluctuation in drug sales of the cth associated drug of the bth class drug; is the distance between the daily sales series of the b-th drug and its c-th associated drug.
[0061] If the sales data of Class B and Class C drugs are closely correlated, and both are only affected by randomness and seasonality, the metric distance between the daily sales series of the two types of drugs will be small, and the sales volume fluctuation degree of Class C drugs will be small, which will ultimately make the correlation deviation value of the two types of drugs smaller.
[0062] S24, obtaining data fluctuation anomalies according to the synchronous change difference weights and associated deviation values of each type of drug and all its associated drugs.
[0063] If a drug has abnormal fluctuations due to statistical errors, the probability that the drugs associated with the drug will also have abnormal fluctuations due to statistical errors is extremely small, and there are often only random and seasonal fluctuations. Therefore, the values of the synchronous change difference weight and the associated deviation value are large. The data fluctuation abnormal value is calculated based on the synchronous change difference weight and the associated deviation value:
[0064] The fusion results of the synchronous change difference weights and associated deviation values of each type of drug and its associated drugs are calculated, and the average of the fusion results of each type of drug and all its associated drugs is used as the data fluctuation abnormal value of each type of drug.
[0065] It should be noted that fusion means that the variable before fusion and the fusion result are in a positive correlation, and the larger the variable, the larger the fusion result. As an embodiment of the present application, the product of the synchronous change difference weight and the associated deviation value of the b-th drug and its c-th associated drug is calculated, and the mean of the product of the b-th drug and all its associated drugs is taken as the data fluctuation abnormal value of the b-th drug.
[0066] As another embodiment of the present application, the fusion result is the sum of the synchronous change difference weights and associated deviation values of each type of drug and its associated drugs.
[0067] If Class B drugs experience abnormal fluctuations due to statistical errors, the probability that drugs associated with Class B drugs also experience abnormal fluctuations due to statistical errors is extremely small, and there are often only random and seasonal fluctuations. Therefore, at this time, the deviation of the daily sales series of Class B drugs and related drugs is large, and the values of the synchronous change difference weights and the associated deviation values are large, and ultimately the data fluctuation abnormalities of Class B drugs in the target month are large.
[0068] The block diagram for obtaining the data fluctuation difference value is as follows: Figure 2 shown.
[0069] S3, obtain an adaptive density threshold according to the abnormal value of data fluctuation of each type of drugs; substitute the adaptive density threshold into the density clustering algorithm to cluster all daily sales of each type of drugs to obtain the stable characteristic sequence of drugs in the target month.
[0070] The data fluctuation anomaly can comprehensively reflect the randomness and seasonal fluctuations of the daily sales data of this type of drugs, indicating that this type of drugs has abnormal fluctuations in the target month. Therefore, the corresponding adaptive density threshold is obtained by combining the daily sales sequence of this type of drugs in the target month: The adaptive density threshold corresponding to the b-th drug is recorded as , ;in, is the adaptive density threshold corresponding to the b-th drug; is the mean of the daily sales series of class b drugs; The data fluctuation abnormal value is for the Class B drugs; is the preset initial density threshold, which is set to 5 in this embodiment. The implementer can set the preset initial density threshold according to the actual data volume; is a linear normalization function; is the rounding function; To preset the control parameters, The value range is .
[0071] in, The role of The value range is adjusted to If there are abnormal sales data of this type of drugs in the target month caused by statistical errors and other factors, the value of the abnormal value of the data fluctuation of this type of drugs is large. Larger, located When the target month is between , a larger density segmentation threshold is required to separate the outliers with smaller fluctuations, and the preset initial density threshold needs to be adjusted to a smaller value. On the contrary, if this type of product only has random and seasonal fluctuations in the target month, the data fluctuation outliers of the drug are smaller, and thus the adaptive density threshold of the drug is smaller. Smaller, located In order to avoid judging normal fluctuation values with large random and seasonal fluctuations as abnormal values, the preset initial density threshold is adjusted to a smaller value.
[0072] In order to analyze the stability of drugs, the stability of a single type of drug in a single month is taken as an example. The daily sales sequence of class b drugs in a single month and the adaptive density threshold are used as the input of the DBSCAN algorithm. The neighborhood radius Eps of the algorithm is set to 5 in this scheme. The algorithm outputs the clustering of the daily sales data of class b drugs in the target month. The cluster with the most elements in all clusters is marked as a normal cluster; the ratio of the number of elements contained in the normal cluster to the number of elements in the daily sales sequence of the class of drugs is recorded as the normal rate of sales data of each class of drugs; the product of the variance of all elements in the normal cluster and the normal rate of sales data of each class of drugs is used as the stability judgment value of each class of drugs; all drugs are sorted according to the stability judgment value to obtain the stable feature sequence of drugs in the target month. In this way, the stability of all drugs in the current region can be obtained. And the DataEase visualization tool is used on the Hadoop big data platform to visualize the stable feature sequence of drugs in region A in the target month.
[0073] See also Figure 3 , Figure 3 1 is a schematic diagram of the structure of an electronic device for extracting key features of large-scale data provided by an embodiment of the present application. In this embodiment, each unit included in the terminal is used to execute each step in an embodiment corresponding to a method for extracting key features of large-scale data. Figure 3 ,The electronic equipment for extracting key features of large scale data includes: a big data acquisition module, a big data analysis module, and a data feature extraction module.
[0074] Big data collection module, used to collect daily sales of various types of drugs in each channel in the target month;
[0075] The big data analysis module is used to obtain the degree of fluctuation of drug sales of each type of drug according to the numerical distribution of daily sales of each type of drug in all channels; obtain all prescriptions of the target month; obtain the related drugs of each type of drug according to the co-occurrence probability of each type of drug and other types of drugs in the prescription; calculate the weight of the synchronous change difference between each type of drug and its related drugs according to the difference in the change of all daily sales of each type of drug and its related drugs and the difference in the degree of drug sales fluctuation; obtain the associated deviation value of each type of drug and its related drugs according to the distance between the daily sales series of each type of drug and its related drugs and the degree of drug sales fluctuation; obtain the data fluctuation abnormal value according to the weight of the synchronous change difference and the associated deviation value of each type of drug and all its related drugs;
[0076] The data feature extraction module is used to obtain an adaptive density threshold according to the abnormal data fluctuation values of each type of drug; the adaptive density threshold is substituted into the density clustering algorithm to cluster all daily sales of each type of drug to obtain a stable feature sequence of the drug in the target month.
[0077] In a third aspect, an embodiment of the present application further provides a large-scale data key feature extraction medium, wherein the data key feature extraction medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above-mentioned large-scale data key feature extraction methods are implemented.
[0078] Through the above description of the implementation method in combination with the accompanying drawings, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.
[0079] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.
Claims
1. A method for extracting key features of large-scale data, characterized in that: The method comprises the following steps: Collect the daily sales volume of each type of medicine in each channel in the target month; According to the numerical distribution of daily sales of various drugs in all channels, the fluctuation degree of drug sales of various drugs is obtained; all prescriptions of the target month are obtained; according to the co-occurrence probability of various drugs with other drugs in the prescription, the related drugs of various drugs are obtained; According to the difference in the daily sales volume of each drug type and its associated drugs and the difference in the degree of fluctuation in drug sales, the weight of the synchronous change difference between each drug type and its associated drugs is calculated; according to the distance between the daily sales series of each drug type and its associated drugs and the degree of fluctuation in drug sales, the associated deviation value of each drug type and its associated drugs is obtained; according to the weight of the synchronous change difference and the associated deviation value of each drug type and its all associated drugs, the data fluctuation abnormal value is obtained; An adaptive density threshold is obtained according to the abnormal value of data fluctuation of each type of drug; the adaptive density threshold is substituted into the density clustering algorithm to cluster all daily sales of each type of drug, and a stable characteristic sequence of drugs in the target month is obtained.
2. A large-scale data key feature extraction method as claimed in claim 1, characterized in that: According to the numerical distribution of daily sales of various drugs in all channels, the fluctuation degree of drug sales of various drugs is obtained, including: The sum of the daily sales volume of each type of drug in all channels is recorded as the daily sales volume of each type of drug; the sequence composed of all daily sales volume of each type of drug in the target month is recorded as the daily sales sequence of each type of drug; the daily sales sequence of each type of drug is fitted with the least square method to obtain the sales curve; and the extreme value points on the sales curve are obtained; Calculate the difference between the daily sales volume corresponding to each extreme point in the daily sales sequence and its previous daily sales volume, and record it as the first difference; calculate the difference between the daily sales volume corresponding to each extreme point in the daily sales sequence and its subsequent daily sales volume, and record it as the second difference; calculate the sum of the first difference and the second difference; and add up the sum of all extreme points on the sales curve, and record it as the degree of fluctuation of drug sales volume of each type of drug.
3. A large-scale data key feature extraction method as claimed in claim 1, characterized in that: According to the co-occurrence probability of each drug and other drugs in the prescription, the associated drugs of each drug are obtained, including: Any type of drug is recorded as the target drug, and the co-occurrence probability of the target drug and other types of drugs except the target drug in all prescriptions of the target month is calculated; the preset number of types of drugs with the highest co-occurrence probability in all prescriptions of the target month are recorded as the associated drugs of the target drug.
4. A large-scale data key feature extraction method as claimed in claim 2, characterized in that: The method for calculating the synchronous change difference weight between each class of drugs and its associated drugs includes: recording the synchronous change difference weight between the b-th class of drugs and its c-th associated drugs as , ; In the formula, Daily sales sequence for category b drugs The variance of The daily sales sequence of the cth associated drug of the bth drug The variance of The degree of fluctuation in drug sales for Category B drugs; The degree of fluctuation in drug sales of the cth associated drug of the bth class drug; is a linear normalization function.
5. A large-scale data key feature extraction method as claimed in claim 2, characterized in that: The method for obtaining the associated deviation value of each associated drug of each type of drug includes: For each type of drug, the distance between the daily sales series of the drug and its associated drugs is calculated; the product of the distance and the degree of fluctuation of the drug sales of each associated drug of each type of drug is used as the association deviation value between the drug and its associated drugs.
6. A large-scale data key feature extraction method as claimed in claim 1, characterized in that: The data fluctuation abnormal value is obtained according to the synchronous change difference weight and associated deviation value of each type of drug and all its associated drugs, including: The fusion results of the synchronous change difference weights and associated deviation values of each type of drug and its associated drugs are calculated, and the average of the fusion results of each type of drug and all its associated drugs is used as the data fluctuation abnormal value of each type of drug.
7. A method for extracting key features of large-scale data as claimed in claim 2, characterized in that: The adaptive density threshold is obtained according to the abnormal value of the data fluctuation of each type of medicine, including: recording the adaptive density threshold corresponding to the b-th type of medicine as , ;in, is the adaptive density threshold corresponding to the b-th drug; is the mean of the daily sales series of class b drugs; The data fluctuation abnormal value is for the Class B drugs; is the preset initial density threshold; is a linear normalization function; is the rounding function; It is the preset control parameter.
8. A large-scale data key feature extraction method as claimed in claim 2, characterized in that: Substituting the adaptive density threshold into the density clustering algorithm to cluster all daily sales of each type of drug, the stable characteristic sequence of drugs in the target month is obtained, including: The adaptive density threshold is used as the density threshold in the density clustering algorithm to cluster the daily sales sequence of each type of drug to obtain each cluster; the cluster containing the most elements in all clusters is marked as a normal cluster; The ratio of the number of elements contained in the normal cluster to the number of elements in the daily sales sequence of the drug class is recorded as the normal rate of sales data of each drug class; the product of the variance of all elements in the normal cluster and the normal rate of sales data of each drug class is used as the stability judgment value of each drug class; All drugs are sorted according to their stability evaluation values to obtain the drug stability characteristic sequence for the target month.
9. An electronic device for extracting key features of large-scale data, characterized in that: The electronic device for extracting key features of large-scale data includes: Big data collection module, used to collect daily sales of various types of drugs in each channel in the target month; The big data analysis module is used to obtain the degree of fluctuation of drug sales of each type of drug according to the numerical distribution of daily sales of each type of drug in all channels; obtain all prescriptions of the target month; obtain the related drugs of each type of drug according to the co-occurrence probability of each type of drug and other types of drugs in the prescription; calculate the weight of the synchronous change difference between each type of drug and its related drugs according to the difference in the change of all daily sales of each type of drug and its related drugs and the difference in the degree of drug sales fluctuation; obtain the associated deviation value of each type of drug and its related drugs according to the distance between the daily sales series of each type of drug and its related drugs and the degree of drug sales fluctuation; obtain the data fluctuation abnormal value according to the weight of the synchronous change difference and the associated deviation value of each type of drug and all its related drugs; The data feature extraction module is used to obtain an adaptive density threshold according to the abnormal data fluctuation values of each type of drug; the adaptive density threshold is substituted into the density clustering algorithm to cluster all daily sales of each type of drug to obtain a stable feature sequence of the drug in the target month.
10. A large-scale data key feature extraction medium, wherein the data key feature extraction medium stores a computer program, characterized in that: When the computer program is executed by a processor, the steps of a large-scale data key feature extraction method as described in any one of claims 1 to 8 are implemented.
