Sample data construction and expansion method and system combined with big data technology
By obtaining the relative characteristic values and expansion coefficients of sales channels in big data technology, screening and expanding data from specific channels, the problem of low effectiveness of random forest algorithm expansion in all channels is solved, and the medical sales analysis is improved Efficiency and accuracy.
Patent Information
- Application Number
- CN202411261752.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-09-10
AI Technical Summary
The existing methods directly expand the pharmaceutical sales data of all sales channels, resulting in the low effectiveness of random forest algorithm for sample data expansion, affecting the efficiency of pharmaceutical sales analysis.
By dividing the preset time period into sub-time periods, obtain the relative characteristic value of each sales channel in each sub-time period, calculate the expansion coefficient based on the similarity of the relative characteristic values, filter out the sales channels that need to be expanded, and use random forests The algorithm augments its sample data.
The effectiveness of the random forest algorithm for sample data expansion has been improved, and the medical sales volume of each sales channel has been accurately analyzed to ensure reasonable medical distribution.
Smart Images

Figure CN118797314B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data prediction technology, and in particular to a sample data construction and expansion method and system combined with big data technology. Background Art
[0002] The development of big data technology is driven by the development of cloud computing, artificial intelligence and other technologies, and has gradually integrated lake-warehouse integration technology and flow-batch integration technology to achieve more efficient data processing and analysis. With the continuous advancement of technology, big data technology has penetrated into the field of pharmaceutical sales. Through big data technology, the sales volume of medicines in various sales channels is analyzed, and then the medicines in various sales channels are reasonably allocated.
[0003] The sales of medicines in different sales channels are different. There may be sales channels with fewer characteristics of changes in medicine sales, which leads to inaccurate analysis of the sales of medicines in various sales channels by big data technology, affecting the reasonable allocation of medicines in different sales channels. In big data technology, sample data expansion can help improve the model's recognition ability for minority classes, especially in unbalanced data sets. Therefore, in existing methods, the sample data of medicine sales in various sales channels is expanded through the random forest algorithm to improve the accuracy of big data technology in analyzing medicine sales in various sales channels. However, in actual situations, not all sales channels have the problem of unclear characteristics of changes in medicine sales. Directly expanding the sample data of medicine sales in all sales channels will result in low effectiveness of random forest algorithm in expanding sample data, affecting the efficiency of medicine sales analysis in various sales channels. Summary of the invention
[0004] In order to solve the technical problem that directly expanding the sample data of pharmaceutical sales of all sales channels will result in low effectiveness of random forest algorithm for sample data expansion, affecting the efficiency of pharmaceutical sales analysis of each sales channel, the purpose of the present invention is to provide a sample data construction and expansion method and system combined with big data technology, and the technical solutions adopted are as follows:
[0005] In a first aspect, an embodiment of the present invention provides a method for constructing and expanding sample data in combination with big data technology, the method comprising the following steps:
[0006] Obtain the daily pharmaceutical sales data of each sales channel within a preset time period as sample data;
[0007] Divide the preset time period into sub-time periods, divide the sample data of each sales channel in each sub-time period into multiple data sequences, and obtain the relative characteristic value of each sales channel in each sub-time period according to the difference between the data sequences of each sales channel and each other sales channel in each sub-time period and the correlation of the sample data;
[0008] According to the similarity between the relative characteristic values corresponding to each sales channel and each other sales channel in a preset time period, the expansion coefficient of each sales channel is obtained; and the sales channels to be expanded are selected based on the expansion coefficient;
[0009] The sample data of each expanded sales channel is expanded through the random forest algorithm.
[0010] Furthermore, the relative characteristic value is obtained by:
[0011] According to the difference between each data series of each sales channel in each sub-time period and each other data series, obtain the special degree value of each data series;
[0012] Arrange the special degree values of each data sequence of each sales channel in each sub-time period according to the time sequence, and obtain the special degree value sequence of each sales channel in each sub-time period;
[0013] According to the difference in the special degree value sequence between each sales channel and each other sales channel in each sub-time period and the correlation of the sample data, the relative characteristic value of each sales channel in each sub-time period is obtained.
[0014] Furthermore, the method for obtaining the special degree value is:
[0015] For any data sequence, the sub-time period of the data sequence is taken as the target time period, the sales channel corresponding to the data sequence is taken as the target channel, and the first feature difference value between the data sequence and each other data sequence of the target channel in the target time period is obtained according to the sample data difference between the data sequence and each other data sequence of the target channel in the target time period;
[0016] The Jaccard coefficients of the data series and each other data series of the target channel in the target time period are obtained in sequence to construct a vector as the relative change vector of the data series;
[0017] The cosine similarity of the relative change vector between the data sequence and each other data sequence of the target channel in the target time period is used as the first feature similarity value between the data sequence and each other data sequence of the target channel in the target time period;
[0018] The special degree value of the data sequence is obtained based on the first feature difference value and the first feature similarity value between the data sequence and each other data sequence of the target channel within the target time period; wherein the first feature difference value and the special degree value are positively correlated, and the first feature similarity value and the special degree value are negatively correlated.
[0019] Furthermore, the method for obtaining the first characteristic difference value is:
[0020] For any data sequence, obtain the mean of all sample data in the data sequence as the reference characteristic value of the data sequence;
[0021] The reference characteristic value difference between the data sequence and each other data sequence of the target channel within the target time period is used as the first characteristic difference value between the data sequence and each other data sequence of the target channel within the target time period.
[0022] Furthermore, the relative characteristic value is obtained by:
[0023] Obtain the Manhattan distance between the special degree value sequence of each sales channel and each other sales channel in each sub-time period as the reference characteristic difference value of each sales channel and each other sales channel in each sub-time period;
[0024] According to the correlation between each sales channel and the sample data of each other sales channel in each sub-time period, the correlation value of each sales channel with each other sales channel in each sub-time period is obtained;
[0025] According to the reference feature difference value and correlation value between each sales channel and each other sales channel in each sub-time period, the relative feature value of each sales channel in each sub-time period is obtained; wherein the first feature value and the correlation value are both positively correlated with the relative feature value.
[0026] Furthermore, the method for obtaining the correlation value is:
[0027] For any sub-time period, the sample data of each sales channel in the sub-time period are arranged in chronological order to obtain the sample data sequence of each sales channel in the sub-time period;
[0028] The absolute value of the Pearson correlation coefficient of the sample data sequence between each sales channel and each other sales channel in the sub-time period is amplified as the correlation degree value of each sales channel with each other sales channel in the sub-time period.
[0029] Furthermore, the method for obtaining the expansion coefficient is:
[0030] The relative characteristic values of each sales channel in all sub-time periods within a preset time period are constructed as a target set, and the probability of each relative characteristic value in the target set appearing in each sales channel is fitted into a curve as a significant curve for each sales channel;
[0031] For any sales channel, obtain the mean of the KL divergence of the significant curves of the sales channel and each other sales channel as the expansion coefficient of the sales channel.
[0032] Furthermore, the method for obtaining the expanded sales channels is:
[0033] The result of normalizing the expansion coefficient is used as the normalized expansion coefficient;
[0034] When the normalized expansion coefficient is less than a preset expansion coefficient threshold, the corresponding sales channel is an expanded sales channel.
[0035] Furthermore, the sample data construction and expansion method combined with big data technology also includes:
[0036] For any expanded sales channel, the ratio of the expansion coefficient of the expanded sales channel to the cumulative expansion coefficients of all expanded sales channels is used as the weight of the expanded sales channel;
[0037] The product of the weight of the expanded sales channel and the number of types of all attributes of the sample data is used as the number of types of attributes of each decision tree of the expanded sales channel in the random forest algorithm;
[0038] According to the sample data of each expanded sales channel and the number of types of attributes of each decision tree, the sample data of each expanded sales channel is expanded by using the random forest algorithm.
[0039] In the second aspect, another embodiment of the present invention provides a sample data construction and expansion system combined with big data technology, the system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, when the processor executes the computer program, the steps of any one of the above methods are implemented.
[0040] The present invention has the following beneficial effects:
[0041] Dividing the preset time period into sub-time periods is conducive to accurately analyzing the distribution of sample data of each sales channel in the local time period, so as to more accurately analyze the sample data characteristics of each sales channel in the subsequent period; further dividing the sample data of each sales channel in each sub-time period into multiple data sequences is conducive to more accurately analyzing the sample data characteristics of each sales channel and avoiding the neglect of some sample data characteristics; then, according to the difference in data sequences between each sales channel and each other sales channel in each sub-time period, and the correlation of sample data, the relative characteristic value of each sales channel in each sub-time period is obtained, accurately reflecting the obvious degree of sample data characteristics of each sales channel in each sub-time period; then, according to the correlation between each sales channel and each other sales channel in the preset time period, the relative characteristic value of each sales channel in each sub-time period is obtained. The similarity of the relative characteristic values can accurately reflect the richness of the sample data characteristics of each sales channel. Therefore, the expansion coefficient of each sales channel is accurately obtained to reflect the similarity of the sample data characteristics between each sales channel and each other sales channel, and indirectly determine the degree to which each sales channel needs to expand the sample data; then the sales channels for expansion are accurately screened out based on the expansion coefficient to improve the efficiency of sample data expansion; finally, the sample data of each expanded sales channel is expanded through the random forest algorithm, which improves the effectiveness of the random forest algorithm in expanding the sample data, and then the sample data in each sales channel after the sample data expansion is analyzed through big data technology, the drug sales situation of each sales channel is accurately analyzed, and the medicines of each sales channel are reasonably allocated. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0043] Figure 1 A schematic flow chart of a sample data construction and expansion method combined with big data technology provided by an embodiment of the present invention;
[0044] Figure 2 A flow chart of a method for obtaining relative characteristic values provided by one embodiment of the present invention;
[0045] Figure 3 A flow chart of a method for obtaining a special degree value provided by an embodiment of the present invention;
[0046] Figure 4 A flowchart of a specific method for obtaining relative characteristic values provided by one embodiment of the present invention;
[0047] Figure 5 A structural diagram of a sample data construction and expansion system combined with big data technology provided by an embodiment of the present invention;
[0048] Figure 6 A schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0049] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the sample data construction expansion method and system proposed by the present invention in combination with the big data technology, its specific implementation method, structure, features and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0050] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0051] The following is a detailed description of a specific solution of a sample data construction and expansion method and system combined with big data technology provided by the present invention with reference to the accompanying drawings.
[0052] Embodiment 1:
[0053] The specific scenario of this embodiment is: in the data analysis process of big data technology, feature analysis is performed on the sample data of each sales channel, and the visual analysis results of the changes in the sales volume of medicines in each sales channel are obtained, and then the medicine sales strategy of each sales channel is reasonably adjusted to ensure that the medicine distribution of each sales channel is reasonable. However, in actual situations, big data technology cannot accurately analyze the sales of medicines for sales channels whose sample data features are not obvious, which leads to the inability to reasonably adjust the medicine sales strategies of each sales channel. Therefore, it is necessary to expand the sample data of sales channels whose sample data features are not obvious, and then accurately analyze the sales volume of medicines in each sales channel. In order to determine the sales channels that need to expand sample data, this embodiment analyzes the differences in sample data changes between each sales channel and each other sales channel in different time periods, and determines the change characteristics of the sample data of each sales channel; at the same time, considering that there is a certain correlation between different sales channels, that is, different sales channels influence each other, therefore, this embodiment further analyzes the correlation between sample data of each sales channel and each other sales channel, accurately obtains the relative characteristic value of each sales channel in different time periods, and then analyzes the similarity of the relative characteristic values between each sales channel and each other sales channel. When the relative characteristic values of a sales channel are more similar to those of other sales channels, it means that the sample data of the sales channel has less rich characteristics under the chain influence of other sales channels, and the sample data of the sales channel needs to be expanded more. Then, according to the similarity of the relative characteristic values between each sales channel and each other sales channel, the expansion coefficient of each sales channel is obtained, and the expanded sales channel is determined. The sample data of each expanded sales channel is expanded by the random forest algorithm, which avoids the problem of low expansion effectiveness of the random forest algorithm, and is conducive to accurate and efficient analysis of the pharmaceutical sales situation of each sales channel, and then reasonable adjustment of the pharmaceutical sales strategy of each sales channel. Among them, the random forest algorithm is a well-known technology and will not be described in detail.
[0054] This invention proposes a sample data construction and expansion method combining big data technology, please refer to Figure 1 , which shows a schematic flow chart of a sample data construction and expansion method combined with big data technology provided by an embodiment of the present invention, the method comprising the following steps:
[0055] Step S1: Obtain daily pharmaceutical sales data for each sales channel within a preset time period as sample data.
[0056] Specifically, the daily pharmaceutical sales data of each sales channel within a preset time period is obtained through big data technology, which is the sample data of each sales channel in the big data technology, wherein the sales channels include Internet e-commerce, DTP (Direct to Patient) pharmacies, traditional pharmacies, community health centers, township health centers, county hospitals, urban hospitals and other sales channels. The sample data is the sum of the number of all drugs sold by each sales channel every day. In this embodiment, the preset time period is set to 3 years. The implementer can set the size of the preset time period according to the actual situation, which is not limited here. In order to accurately analyze the pharmaceutical sales situation of each sales channel, this embodiment first arranges the sample data of each sales channel within the preset time period in the order of time from first to last, constructs the overall sample data sequence of each sales channel, and then uses the data cleaning algorithm to pre-process each overall sample data sequence, detects and processes the missing values and outliers in each overall sample data sequence, ensures that there is no error in the sample data, and is conducive to the subsequent accurate analysis of the sample data characteristics of each sales channel. Among them, the data cleaning algorithm is a well-known technology and will not be repeated.
[0057] Step S2: Divide the preset time period into sub-time periods, divide the sample data of each sales channel in each sub-time period into multiple data sequences, and obtain the relative characteristic value of each sales channel in each sub-time period based on the difference in data sequences between each sales channel and each other sales channel in each sub-time period and the correlation of the sample data.
[0058] In actual situations, the sales of medicines in different sales channels are affected by the region, sales methods, and medicine supply and marketing, resulting in large differences in the relative characteristics of sample data in different sales channels. For example, in a certain local time period, due to the existence of too many bad reviews of online stores selling medicines in the Internet e-commerce sales channel, it will affect the sales of medicines in offline sales channels, resulting in a chain effect, that is, the sales of medicines in offline sales channels will increase, and the degree of chain effect of medicine sales in different sales channels is different. In order to accurately analyze the characteristics of sample data of each sales channel under the influence of chain effects, and then accurately determine the sales channels that need to expand sample data, therefore, this embodiment first divides the preset time period into multiple sub-time periods, analyzes the sample data of each sales channel in each sub-time period, and then more accurately analyzes the characteristics of sample data of each sales channel under the influence of chain effects. Among them, this embodiment sets the sub-time period to one month and defaults to 30 days for one month. The implementer can set the size of the sub-time period according to the actual situation, which is not limited here.
[0059] In order to more accurately analyze the sample data characteristics of each sales channel in each sub-time period, this embodiment further divides the sample data of each sales channel in each sub-time period into multiple data sequences, which is conducive to more accurately analyzing the subtle changes in the sample data of each sales channel and more accurately obtaining the sample data characteristics of each sales channel in each sub-time period. In this embodiment, the sample data of each sales channel in each sub-time period is evenly divided into 6 data sequences. The implementer can set the number of data sequences divided by the sample data of each sales channel in each sub-time period according to the actual situation, which is not limited here. In a certain sub-time period, when the changes in the sample data in the data sequence of a certain sales channel and each other sales channel in the same time period are more different, and the degree of correlation between the sample data of the sales channel and each other sales channel is greater, it means that the sample data of the sales channel in the sub-time period still has obvious relative characteristics under the influence of a larger chain, which indirectly reflects that the sample data characteristics of the sales channel in the sub-time period are more obvious. Therefore, this embodiment obtains the relative characteristic value of each sales channel in each sub-time period according to the difference in the data sequence between each sales channel and each other sales channel in each sub-time period and the correlation of the sample data. The larger the relative characteristic value, the more obvious the sample data characteristics of the corresponding sales channel are in the corresponding sub-time period.
[0060] Preferably, in some possible implementations of this embodiment, the method for obtaining the relative characteristic value can be found in Figure 2 , which shows a flow chart of a method for obtaining a relative characteristic value provided by an embodiment of the present invention, the method comprising the following steps:
[0061] Step S201: Obtain the special degree value of each data sequence according to the difference between each data sequence of each sales channel in each sub-time period and each other data sequence.
[0062] In order to analyze the characteristics of the sample data in each data sequence, this embodiment compares each data sequence of each sales channel in each sub-time period with each other data sequence, accurately analyzes the distribution of the sample data in each data sequence of each sales channel in each sub-time period, and then obtains the special degree value of each data sequence according to the difference between each data sequence of each sales channel in each sub-time period and each other data sequence. The larger the special degree value, the more significant the distribution of the sample data in the corresponding data sequence in the corresponding sub-time period, which is conducive to the subsequent accurate analysis of the distribution difference of the sample data between different sales channels in each sub-time period.
[0063] Preferably, in some possible implementations of this embodiment, the method for obtaining the special degree value can be found in Figure 3, which shows a flow chart of a method for obtaining a special degree value provided by an embodiment of the present invention, the method comprising the following steps:
[0064] For any data sequence, the sub-time period of the data sequence is taken as the target time period, and the sales channel corresponding to the data sequence is taken as the target channel.
[0065] Step S201 - 1 : Obtain a first feature difference value.
[0066] First, based on the sample data difference between the data sequence and each other data sequence of the target channel in the target time period, the first feature difference value between the data sequence and each other data sequence of the target channel in the target time period is obtained. The larger the first feature difference value, the more different the sample data between the data sequence and other corresponding data sequences of the target channel in the target time period is, which indirectly indicates that the sample data in the data sequence is more special.
[0067] Preferably, in some possible implementations of the present embodiment, the method for obtaining the first characteristic difference value is: for any data sequence, obtaining the mean of all sample data in the data sequence as the reference characteristic value of the data sequence; taking the absolute value of the difference between the reference characteristic values of the data sequence and each other data sequence of the target channel within the target time period as the first characteristic difference value between the data sequence and each other data sequence of the target channel within the target time period.
[0068] Step S201 - 2: Obtain the first feature similarity value.
[0069] In order to more accurately obtain the difference between the data sequence and each other data sequence of the target channel in the target time period, and then more accurately analyze the special degree of the data sequence, this embodiment further obtains the Jaccard coefficient of the data sequence and each other data sequence of the target channel in the target time period, and arranges the obtained Jaccard coefficients in the order of the corresponding other data sequences from first to last in time series, and constructs a vector as the relative change vector of the data sequence. For example, the 6 data sequences of the u-th sales channel in the f-th sub-time period are respectively , , , , and , where we get The method for obtaining the relative change vector of is as follows: First obtain and The Jaccard coefficient is ; and The Jaccard coefficient is ; and The Jaccard coefficient is ; and The Jaccard coefficient is ; and The Jaccard coefficient is ; Then the Jaccard coefficient is calculated according to the corresponding , , , , Arrange them in chronological order from first to last, that is , , , , ,therefore, The relative change vector is . Among them, the method for obtaining the Jaccard coefficient is a well-known technology and will not be described in detail. The larger the Jaccard coefficient, the greater the difference between the sample data in the corresponding two data sequences. In order to accurately obtain the particularity of the data sequence within the target time period, this embodiment obtains the cosine similarity of the relative change vector of the data sequence and each other data sequence of the target channel within the target time period as the first feature similarity value of the data sequence and each other data sequence of the target channel within the target time period; the smaller the first feature similarity value, the more different the sample data in the data sequence is from the sample data in other data sequences of the target channel within the target time period, and the more special the sample data in the data sequence is.
[0070] Step S201 - 3: Obtain the special degree value.
[0071] It is known that the larger the first feature difference value is, the smaller the first feature similarity value is, indicating that the sample data in the data sequence is more special; therefore, this embodiment obtains the special degree value of the data sequence based on the first feature difference value and the first feature similarity value of the data sequence and each other data sequence of the target channel in the target time period; wherein the first feature difference value is positively correlated with the special degree value, and the first feature similarity value is negatively correlated with the special degree value. Therefore, the calculation formula of the special degree value is: , where The special degree value of the t-th data sequence of the u-th sales channel in the f-th sub-time period; M is the total number of data sequences of the u-th sales channel in the f-th sub-time period; is the reference characteristic value of the tth data sequence of the uth sales channel in the fth sub-time period; is the reference characteristic value of the mth data sequence of the uth sales channel in the fth sub-time period; is the first feature difference value between the t-th data sequence and the m-th data sequence of the u-th sales channel in the f-th sub-time period; is the absolute value function; is the first feature similarity value between the t-th data sequence and the m-th data sequence of the u-th sales channel in the f-th sub-time period; It is the first preset constant, which is greater than 0.
[0072] This embodiment will Set to 0.01 to avoid the denominator being 0. The implementer can set it according to the actual situation. The size of is not limited here. So far, the special degree value of each data sequence is obtained.
[0073] Step S202: Arrange the special degree values of each data sequence of each sales channel in each sub-time period according to the time sequence, and obtain the special degree value sequence of each sales channel in each sub-time period.
[0074] In order to facilitate the subsequent accurate analysis of the distribution differences of sample data between different sales channels in each sub-time period, and then accurately obtain the sample data characteristics of each sales channel in each sub-time period, this embodiment arranges the special degree values of each data sequence of each sales channel in each sub-time period according to the time sequence of the corresponding data sequence from first to last, and obtains the special degree value sequence of each sales channel in each sub-time period. In other embodiments, the special degree values of each data sequence of each sales channel in each sub-time period can be arranged according to other uniformly specified orders to obtain the special degree value sequence of each sales channel in each sub-time period, which is not limited here.
[0075] Step S203: Obtain the relative characteristic value of each sales channel in each sub-time period according to the difference in the special degree value sequence between each sales channel and each other sales channel in each sub-time period and the association of the sample data.
[0076] In order to accurately analyze the sample data characteristics of each sales channel under the chain influence of different sales channels in each sub-time period, this embodiment first analyzes the difference in the special degree value sequence between each sales channel and each other sales channel in each sub-time period. When the difference in the special degree value sequence between a sales channel and each other sales channel in a sub-time period is greater, it means that the sample data characteristics of the sales channel are more obvious in the sub-time period. Considering that there is a correlation between different sales channels, in order to more accurately obtain the obvious degree of the sample data characteristics of each sales channel in each sub-time period, this embodiment further obtains the correlation degree of the sample data between each sales channel and each other sales channel in each sub-time period. When the correlation degree of the sample data between a sales channel and each other sales channel in a sub-time period is greater, it means that the sales channel is more affected by the chain influence of other sales channels in the sub-time period. When the difference in the special degree value sequence between a sales channel and each other sales channel in a sub-time period is greater and the correlation degree is greater, it can more accurately explain that the sample data characteristics of the sales channel are more obvious in the sub-time period. Therefore, this embodiment obtains the relative characteristic value of each sales channel in each sub-time period according to the difference in the special degree value sequence between each sales channel and each other sales channel in each sub-time period and the correlation of the sample data. The larger the relative characteristic value, the more significant the sample data feature of the corresponding sales channel is in the corresponding sub-time period.
[0077] Preferably, in some possible implementations of this embodiment, the specific method for obtaining the relative characteristic value can be found in Figure 4 , which shows a flowchart of a specific method for obtaining relative characteristic values provided by an embodiment of the present invention, the method comprising the following steps:
[0078] Step S203 - 1 : Obtain the Manhattan distance between the special degree value sequence of each sales channel and each other sales channel in each sub-time period as the reference feature difference value between each sales channel and each other sales channel in each sub-time period.
[0079] The larger the reference feature difference value is, the more inconsistent the distribution of the sample data of the two sales channels in the corresponding sub-time period is, which indirectly reflects that the sample data characteristics of the two sales channels under the chain effect are more different. When the Manhattan distance of the special degree value sequence between a sales channel and each other sales channel in a sub-time period is larger, it means that the sample data characteristics of the sales channel are more obvious in the sub-time period. Among them, the method of obtaining the Manhattan distance is a well-known technology and will not be repeated here.
[0080] Step S203 - 2 : According to the association between the sample data of each sales channel and each other sales channel in each sub-time period, obtain the correlation value of each sales channel with each other sales channel in each sub-time period.
[0081] It is known that there are correlations between different sales channels. In order to more accurately obtain the relative characteristic value of each sales channel in each sub-time period, this embodiment further obtains the correlation value between each sales channel and each other sales channel in each sub-time period. The larger the correlation value, the greater the chain effect between the corresponding two sales channels in the corresponding sub-time period. When the correlation value of a sales channel with each other sales channel in a sub-time period is larger, if the reference characteristic difference value between the sales channel and each other sales channel in the sub-time period is still larger, it is more accurate to explain that the sample data characteristics of the sales channel in the sub-time period are more obvious. Furthermore, this embodiment obtains the correlation value of each sales channel with each other sales channel in each sub-time period based on the correlation between the sample data of each sales channel and each other sales channel in each sub-time period.
[0082] Preferably, in some possible implementations of this embodiment, the method for obtaining the correlation value is: for any sub-time period, the sample data of each sales channel in the sub-time period are arranged in time order from first to last, and the sample data sequence of each sales channel in the sub-time period is obtained; the result of amplifying the absolute value of the Pearson correlation coefficient of the sample data sequence of each sales channel and each other sales channel in the sub-time period is used as the correlation value of each sales channel with each other sales channel in the sub-time period. In this embodiment, the absolute value of the Pearson correlation coefficient is used as the exponent of an exponential function with a natural constant as the base to amplify the absolute value of the Pearson correlation coefficient. In other embodiments, the absolute value of the Pearson correlation coefficient can be amplified by adding it to a specified positive number, which is not limited here. Among them, the method for obtaining the Pearson correlation coefficient is a well-known technology and will not be repeated here. At this point, the correlation value of each sales channel with each other sales channel in each sub-time period is obtained.
[0083] Step S203-3: Obtain the relative characteristic value of each sales channel in each sub-time period according to the reference characteristic difference value and correlation value between each sales channel and each other sales channel in each sub-time period; wherein the first characteristic value and the correlation value are both positively correlated with the relative characteristic value.
[0084] When the reference feature difference value and correlation value of a sales channel with each other sales channel in a sub-time period are larger, it means that the sample data feature of the sales channel is more obvious in the sub-time period, that is, the relative feature value of the sales channel in the sub-time period is larger. Therefore, this embodiment obtains the relative feature value of each sales channel in each sub-time period according to the reference feature difference value and correlation value of each sales channel with each other sales channel in each sub-time period; wherein the first feature value and the correlation value are both positively correlated with the relative feature value. Therefore, the calculation formula of the relative feature value is: ; In the formula, is the relative characteristic value of the vth sales channel in the fth sub-time period; N is the total number of sales channels; is the reference characteristic difference value between the vth sales channel and the nth sales channel in the fth sub-time period; is the Pearson correlation coefficient between the vth sales channel and the nth sales channel in the fth sub-period; is the correlation value between the vth sales channel and the nth sales channel in the fth sub-time period; e is a natural constant; is an absolute value function. In other embodiments, Get , which is not limited here.
[0085] At this point, the relative characteristic value of each sales channel in each sub-time period is obtained.
[0086] Step S3: according to the similarity between the relative characteristic values corresponding to each sales channel and each other sales channel within a preset time period, obtain the expansion coefficient of each sales channel; and select the expanded sales channels based on the expansion coefficient.
[0087] Specifically, in order to determine the sales channels that need to be expanded for sample data, this embodiment analyzes the similarity between the relative characteristic values corresponding to each sales channel and each other sales channel within the preset time period. When the relative characteristic values of a sales channel and each other sales channel within the preset time period are more similar, it means that the sample data characteristics of the sales channel are less obvious, that is, under the chain influence of different sales channels, the sample data of the sales channel has less rich characteristics. In order to enable big data technology to accurately analyze the drug sales situation of the sales channel, the sample data of the sales channel needs to be expanded so that the sample data characteristics of the sales channel are significant. Therefore, this embodiment obtains the expansion coefficient of each sales channel based on the similarity between the relative characteristic values corresponding to each sales channel and each other sales channel within the preset time period, and then screens out the expanded sales channels that need to be expanded for sample data expansion.
[0088] Preferably, in some possible implementations of this embodiment, the method for obtaining the expansion coefficient is: the relative characteristic values of each sales channel in all sub-time periods within a preset time period are constructed as a target set, and it should be noted that there are no identical elements in the target set. The probability of each relative characteristic value in the target set appearing in each sales channel is fitted into a curve by the least squares method, which is used as the significant curve of each sales channel. Among them, the method of fitting the curve by the least squares method is a well-known technology and will not be repeated. For any sales channel, the mean of the KL divergence of the significant curves of the sales channel and each other sales channel is obtained as the expansion coefficient of the sales channel. Among them, the method for obtaining the KL divergence is a well-known technology and will not be repeated. The smaller the KL divergence, the more similar the significant curves of the corresponding two sales channels are. Therefore, when the expansion coefficient is smaller, the sample data characteristics of the corresponding sales channel are less obvious, that is, the higher the degree of lack of sample data characteristics of the corresponding sales channel under the chain effect, the more the sample data of the corresponding sales channel needs to be expanded.
[0089] It is known that the smaller the expansion coefficient is, the more the sample data of the corresponding sales channel needs to be expanded. Therefore, this embodiment normalizes the expansion coefficient to obtain the normalized expansion coefficient of each sales channel. When the normalized expansion coefficient is less than the preset expansion coefficient threshold, the corresponding sales channel is an expanded sales channel. This embodiment sets the preset expansion coefficient threshold to 0.7. The implementer can set the preset expansion coefficient threshold according to the actual situation, which is not limited here. At this point, the expanded sales channel is screened out.
[0090] Step S4: Expand the sample data of each expanded sales channel by using the random forest algorithm.
[0091] Specifically, in order to more effectively expand the sample data of each expanded sales channel through the random forest algorithm, this embodiment obtains all attributes of the sample data, wherein all attributes of the sample data are: the mean, variance, coefficient of variation and number of drug types sold of all types of drugs sold daily, the range of all types of drug sales daily, the proportion of the number of drug types sold daily in the number of all drug types sold, the proportion of each drug sales volume in the sales volume of all drugs sold daily, the proportion of daily pharmaceutical sales data in the total pharmaceutical sales data under all channels, and the total amount of daily pharmaceutical sales. It is known that the smaller the expansion coefficient, the less obvious the characteristics of the sample data in the corresponding expanded sales channel. In order to highlight the characteristics of the sample data in the corresponding expanded sales channel, in the process of expanding the sample data of the corresponding expanded sales channel through the random forest algorithm, the attribute types of the decision tree in the random forest algorithm should be fewer, so that the effective characteristics of the sample data expansion of the corresponding expanded sales channel are more obvious. Therefore, this embodiment obtains the number of attribute types of each decision tree of each expanded sales channel in the random forest algorithm, that is, the number of attribute types appearing in each decision tree, through the expansion coefficient of each expanded sales channel and the number of types of all attributes of the sample data, and then efficiently and effectively expands each expanded sales channel through the random forest algorithm.
[0092] Preferably, in some possible implementations of this embodiment, the method for obtaining the number of attribute types of each decision tree of each expanded sales channel in the random forest algorithm is as follows: for any expanded sales channel, the ratio of the expansion coefficient of the expanded sales channel to the cumulative result of the expansion coefficients of all expanded sales channels is used as the weight of the expanded sales channel; the product of the weight of the expanded sales channel and the number of attribute types of all sample data is used as the number of attribute types of each decision tree of the expanded sales channel in the random forest algorithm. The calculation formula for the number of attribute types of each decision tree of each expanded sales channel in the random forest algorithm is: ; In the formula, is the number of attribute types of each decision tree in the random forest algorithm for the jth expanded sales channel; G is the number of attribute types of all sample data; is the expansion coefficient of the jth expanded sales channel; To expand the total number of sales channels; is the expansion coefficient of the i-th expanded sales channel; is the weight of the jth expanded sales channel. So far, the number of attribute types of each decision tree in the random forest algorithm for each expanded sales channel is obtained. Among them, the attribute types of each decision tree are randomly selected.
[0093] In this embodiment, W sample data are extracted from the sample data of each expanded sales channel with replacement each time to construct each decision tree of each expanded sales channel in the random forest algorithm, wherein the size of W is half of the total number of all sample data of each expanded sales channel in the preset time period. If half of the total number of all sample data of each expanded sales channel in the preset time period is a decimal, half of the total number of all sample data of each expanded sales channel in the preset time period is rounded up to obtain W. Then, according to the sample data of each expanded sales channel and the number of types of attributes of each decision tree, the sample data of each expanded sales channel is expanded by the random forest algorithm until the number of sample data expanded by each expanded sales channel is 2W, and then the sample data expansion of the expanded sales channel is stopped.
[0094] Finally, the sample data of each expanded sales channel is analyzed through big data technology, and the drug sales situation of each sales channel is accurately analyzed, so as to reasonably adjust the drug sales strategy of each sales channel.
[0095] So far, the present invention is completed.
[0096] In summary, the present embodiment obtains sample data of sales channels; divides the sample data of each sales channel in a sub-time period into a data sequence, obtains the relative characteristic value of each sales channel according to the difference in the data sequence between each sales channel and each other sales channel in the sub-time period, and the correlation of the sample data; obtains the expansion coefficient of each sales channel according to the similarity of the relative characteristic values corresponding to each sales channel and each other sales channel, and then screens out the expanded sales channels; expands the sample data of the expanded sales channels by the random forest algorithm. The present invention accurately screens out the expanded sales channels by obtaining the expansion coefficient, improves the efficiency of expanding the sample data of the sales channels, and at the same time improves the effectiveness of the random forest algorithm in expanding the sample data.
[0097] Embodiment 2:
[0098] The present invention also proposes a sample data construction and expansion system combined with big data technology, please refer to Figure 5 , which shows a structure diagram of a sample data construction and expansion system combined with big data technology provided by an embodiment of the present invention. The system includes: a data acquisition module 10, a relative feature value acquisition module 20, an expansion sales channel acquisition module 30 and an expansion module 40.
[0099] The data acquisition module 10 is used to acquire the daily pharmaceutical sales data of each sales channel within a preset time period as sample data.
[0100] The relative characteristic value acquisition module 20 is used to divide the preset time period into sub-time periods, divide the sample data of each sales channel in each sub-time period into multiple data sequences, and obtain the relative characteristic value of each sales channel in each sub-time period based on the difference in data sequences between each sales channel and each other sales channel in each sub-time period and the correlation between the sample data.
[0101] The expanded sales channel acquisition module 30 is used to obtain the expansion coefficient of each sales channel according to the similarity between the relative characteristic values corresponding to each sales channel and each other sales channel within a preset time period; and select the expanded sales channels based on the expansion coefficient.
[0102] The expansion module 40 is used to expand the sample data of each expanded sales channel by using a random forest algorithm.
[0103] It should be noted that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the sample data construction and expansion system combined with big data technology and the sample data construction and expansion method embodiment combined with big data technology provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0104] Embodiment 3:
[0105] The present invention also proposes a sample data construction and expansion device combined with big data technology, the device includes a memory and a processor, wherein the memory stores an executable program code, and the processor is used to call and execute the executable program code to execute a sample data construction and expansion method combined with big data technology provided in the embodiment of the present application. The device can be a chip, a component or a module, and the chip may include a connected processor and a memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a sample data construction and expansion method combined with big data technology provided in the above embodiment.
[0106] In addition, the present application embodiment also protects a computer device, see Figure 6 The computer device includes a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and running on the processor 402, wherein when the processor 402 executes the computer program 403, the computer device can execute any of the sample data construction and expansion methods combined with big data technology introduced above.
[0107] Embodiment 4:
[0108] This embodiment also provides a computer-readable storage medium, which stores computer program code. When the computer program code runs on a computer, the computer executes the above-mentioned related method steps to implement a sample data construction and expansion method combined with big data technology provided in the above embodiment.
[0109] Embodiment 5:
[0110] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a sample data construction and expansion method combined with big data technology provided in the above embodiment.
[0111] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.
[0112] It should be noted that the sequence of the above embodiments of the present invention is for description only and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0113] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
Claims
1. A sample data construction and expansion method combined with big data technology, characterized in that: The method comprises the following steps: Obtain the daily pharmaceutical sales data of each sales channel within a preset time period as sample data; Divide the preset time period into sub-time periods, divide the sample data of each sales channel in each sub-time period into multiple data sequences, and obtain the relative characteristic value of each sales channel in each sub-time period according to the difference between the data sequences of each sales channel and each other sales channel in each sub-time period and the correlation of the sample data; According to the similarity between the relative characteristic values corresponding to each sales channel and each other sales channel in a preset time period, the expansion coefficient of each sales channel is obtained; and the sales channels to be expanded are selected based on the expansion coefficient; The sample data of each expanded sales channel is expanded through the random forest algorithm.
2. A sample data construction and expansion method combined with big data technology as claimed in claim 1, characterized in that: The method for obtaining the relative characteristic value is: According to the difference between each data series of each sales channel in each sub-time period and each other data series, obtain the special degree value of each data series; Arrange the special degree values of each data sequence of each sales channel in each sub-time period according to the time sequence, and obtain the special degree value sequence of each sales channel in each sub-time period; According to the difference in the special degree value sequence between each sales channel and each other sales channel in each sub-time period and the correlation of the sample data, the relative characteristic value of each sales channel in each sub-time period is obtained.
3. A sample data construction and expansion method combined with big data technology as claimed in claim 2, characterized in that: The method for obtaining the special degree value is: For any data sequence, the sub-time period of the data sequence is taken as the target time period, the sales channel corresponding to the data sequence is taken as the target channel, and the first feature difference value between the data sequence and each other data sequence of the target channel in the target time period is obtained according to the sample data difference between the data sequence and each other data sequence of the target channel in the target time period; The Jaccard coefficients of the data series and each other data series of the target channel in the target time period are obtained in sequence to construct a vector as the relative change vector of the data series; The cosine similarity of the relative change vector between the data sequence and each other data sequence of the target channel in the target time period is used as the first feature similarity value between the data sequence and each other data sequence of the target channel in the target time period; The special degree value of the data sequence is obtained based on the first feature difference value and the first feature similarity value between the data sequence and each other data sequence of the target channel within the target time period; wherein the first feature difference value and the special degree value are positively correlated, and the first feature similarity value and the special degree value are negatively correlated.
4. A sample data construction and expansion method in combination with big data technology as claimed in claim 3, characterized in that: The method for obtaining the first characteristic difference value is: For any data sequence, obtain the mean of all sample data in the data sequence as the reference characteristic value of the data sequence; The reference characteristic value difference between the data sequence and each other data sequence of the target channel within the target time period is used as the first characteristic difference value between the data sequence and each other data sequence of the target channel within the target time period.
5. The method for constructing and expanding sample data in combination with big data technology as claimed in claim 2, characterized in that: The method for obtaining the relative characteristic value is: Obtain the Manhattan distance between the special degree value sequence of each sales channel and each other sales channel in each sub-time period as the reference characteristic difference value of each sales channel and each other sales channel in each sub-time period; According to the correlation between each sales channel and the sample data of each other sales channel in each sub-time period, the correlation value of each sales channel with each other sales channel in each sub-time period is obtained; According to the reference feature difference value and correlation value between each sales channel and each other sales channel in each sub-time period, the relative feature value of each sales channel in each sub-time period is obtained; wherein the first feature value and the correlation value are both positively correlated with the relative feature value.
6. A sample data construction and expansion method combined with big data technology as claimed in claim 5, characterized in that: The method for obtaining the correlation value is: For any sub-time period, the sample data of each sales channel in the sub-time period are arranged in chronological order to obtain the sample data sequence of each sales channel in the sub-time period; The absolute value of the Pearson correlation coefficient of the sample data sequence between each sales channel and each other sales channel in the sub-time period is amplified as the correlation degree value of each sales channel with each other sales channel in the sub-time period.
7. The method for constructing and expanding sample data in combination with big data technology as claimed in claim 1, characterized in that: The method for obtaining the expansion coefficient is: The relative characteristic values of each sales channel in all sub-time periods within a preset time period are constructed as a target set, and the probability of each relative characteristic value in the target set appearing in each sales channel is fitted into a curve as a significant curve for each sales channel; For any sales channel, obtain the mean of the KL divergence of the significant curves of the sales channel and each other sales channel as the expansion coefficient of the sales channel.
8. The method for constructing and expanding sample data in combination with big data technology as claimed in claim 1, characterized in that: The method for obtaining the expanded sales channels is: The result of normalizing the expansion coefficient is used as the normalized expansion coefficient; When the normalized expansion coefficient is less than a preset expansion coefficient threshold, the corresponding sales channel is an expanded sales channel.
9. The method for constructing and expanding sample data in combination with big data technology as claimed in claim 1, characterized in that: The sample data construction and expansion method combined with big data technology also includes: For any expanded sales channel, the ratio of the expansion coefficient of the expanded sales channel to the cumulative expansion coefficients of all expanded sales channels is used as the weight of the expanded sales channel; The product of the weight of the expanded sales channel and the number of types of all attributes of the sample data is used as the number of types of attributes of each decision tree of the expanded sales channel in the random forest algorithm; According to the sample data of each expanded sales channel and the number of types of attributes of each decision tree, the sample data of each expanded sales channel is expanded by using the random forest algorithm.
10. A sample data construction and expansion system combined with big data technology, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When executing the computer program, the processor implements the steps of a sample data construction and expansion method combined with big data technology as described in any one of claims 1 to 9.
