A supply chain data management method and system based on big data

Abstract

The present invention discloses a supply chain data management method and system based on big data, and relates to the technical field of supply chain management. By acquiring historical transaction time series data of merchants to be supervised, extracting daily transaction feature information in the transaction data for normalization processing, using the normalized daily transaction feature information time series data as the input sequence of the LSTM neural network, using the neural network to obtain the first predicted value of the daily transaction feature information at the current moment, using the normalized daily transaction feature information time series data to build a statistical prediction model, and calculating the second predicted value of the daily transaction feature information at the current moment, weighting the first predicted value and the second predicted value to obtain the weighted predicted value of the daily transaction feature information at the current moment, setting the prediction allowable error, calculating the predicted value confidence interval of the daily transaction feature information, and determining the abnormality of the daily transaction feature information at the current moment according to the obtained confidence interval.

Description

Technical Field

[0001] The present invention relates to the technical field of supply chain management, and in particular to a supply chain data management method and system based on big data. Background Art

[0002] In the context of big data, digital transformation is an inevitable trend for traditional industries to achieve high-quality development. The retail industry is closely related to the Internet and is a leading field for digital transformation. The use of big data has brought great changes to the development of the retail industry and has promoted the optimization of the retail structure to a certain extent. However, in the era of big data, due to the huge amount of data generated, the possibility of false information being mixed in is high and difficult to detect. Too much false information will greatly affect the accuracy of the data.

[0003] In the supply chain management of the retail industry, with the continuous increase in the number of enterprises, especially the continuous expansion of the scale of small and micro businesses, there are more and more risky businesses that engage in illegal transactions and fraudulent behaviors, which undoubtedly puts higher requirements on the supervisory responsibilities of regulators. The traditional supply chain supervision method is no longer suitable for the management of retail supply chain data in today's big data era, making it impossible to timely identify abnormal or even illegal transactions from massive data, resulting in a lack of supervision and affecting the rights and interests of consumers. To this end, we propose a supply chain data management method and system based on big data. Summary of the invention

[0004] The main purpose of the present invention is to provide a supply chain data management method and system based on big data, which can effectively solve the problems in the background technology.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A supply chain data management method based on big data, comprising:

[0007] Obtain the historical transaction time series data of the merchant to be regulated, extract the daily transaction feature information in the transaction data, and perform normalization processing on the acquired daily transaction feature information data, wherein the daily transaction feature information includes daily transaction amount data, daily transaction number data, and daily transaction total volume data; the normalization processing formula is:

[0008]

[0009] In the formula, is the normalized value of the daily transaction characteristic information data described in item j; j is the sampling value of the daily transaction characteristic information data described in item j; jmin 、x jmaxare respectively the minimum and maximum values ​​in the historical transaction time series data of the daily transaction characteristic information described in item j;

[0010] The normalized daily transaction feature information time series data is used as the input sequence of the LSTM neural network, and the neural network is used to obtain the first predicted value o of the daily transaction feature information of the jth item at the current moment. 1j ;

[0011] The statistical prediction model is constructed using the normalized daily transaction feature information time series data, and the second prediction value o of the daily transaction feature information of the jth item at the current moment is calculated. 2j , where the expression of the statistical prediction model is:

[0012]

[0013] In the formula, It is represented as the normalized value of the t-th sampling value of the daily transaction characteristic information described in the j-th item; It is represented as the normalized value of the tp-th sampling value of the daily transaction feature information described in the j-th item; is represented by the normalized value of the qth sample value of the daily transaction feature information described in the jth item; λ is a constant coefficient, and λ∈(0,1); m and n are both positive integers, and m+n=t; It is represented as the normalized value of the first sample value of the daily transaction characteristic information described in the j-th item;

[0014] The first predicted value o will be obtained 1j With the second predicted value o 2j Perform weighted processing to obtain the weighted prediction value o of the daily transaction feature information described in item j at the current moment j , the calculation formula is: j =ko 1j +(1-k)o 2j , where k is the weight coefficient of the predicted value, and k∈(0,1), the process of determining the weight coefficient k of the predicted value includes the following steps:

[0015] Obtain the historical transaction time series data subsequence that is closest to the current time and has a time length of T in the historical transaction time series data;

[0016] The data in the subsequence is used as the validation sample. The first prediction value obtained by the LSTM neural network and the second prediction value obtained by the statistical prediction model are compared with the validation sample to obtain the classification result of the validation sample. The classification principle is:

[0017] When the validation sample is normal data:

[0018] If the predicted value - validation sample value ≤ the prediction allowable error is d j , then it is classified as TP;

[0019] If the predicted value - validation sample value > the prediction allowable error is d j , then it is classified as FP;

[0020] When the validation sample is abnormal data:

[0021] If the predicted value - validation sample value ≤ the prediction allowable error is d j , then it is classified as FN;

[0022] If the predicted value - validation sample value > the prediction allowable error is d j , then it is classified as TN;

[0023] According to the classification results of the verification samples, the prediction accuracy A of the LSTM neural network and the statistical prediction model are calculated respectively. The calculation formula is: A = (TP + TN) / (TP + FP + FN + TN);

[0024] The weight coefficient k of the prediction value is calculated with the obtained accuracy, and the calculation formula is: k = A1 / (A1+A2), where A1 and A2 are the prediction accuracy values ​​of the LSTM neural network and the statistical prediction model respectively obtained by using the subsequence data of the historical transaction time series data with a time length of T as the verification sample;

[0025] Set the prediction tolerance to d j , calculate the confidence interval of the predicted value of the daily transaction characteristic information described in item j [o jmin ,o jmax ], where o jmin , o jmax are the minimum and maximum values ​​of the confidence interval of the predicted value of the daily transaction characteristic information described in item j, respectively, and the calculation formula is: jmin =o j -d j , o jmax =o j +d j ;

[0026] The abnormality of the daily transaction characteristic information described in the jth item at the current moment is determined according to the obtained confidence interval, wherein the abnormality determination principle is:

[0027] If the daily transaction characteristic information data sampling value described in item j at the current moment is within the confidence interval [o jmin ,o jmax ], it is judged as normal data; otherwise, it is judged as abnormal data.

[0028] A supply chain data management system based on big data, including a data acquisition module, a feature information extraction module, a data processing module, a neural network module, a data statistics prediction module, and an abnormal data determination module;

[0029] The data acquisition module is used to acquire the historical transaction time series data of the merchant to be supervised;

[0030] The feature information extraction module is used to extract daily transaction feature information from the transaction data, wherein the daily transaction feature information includes daily transaction amount data, daily transaction number data, and daily transaction total volume data;

[0031] The data processing module is used to normalize the acquired daily transaction characteristic information data;

[0032] The neural network module is used to construct an LSTM neural network, and use the neural network to obtain the first predicted value o of the daily transaction feature information of the jth item at the current moment. 1j , wherein the LSTM neural network uses the normalized daily transaction feature information time series data as an input sequence;

[0033] The data statistical prediction module is used to construct a statistical prediction model using the normalized daily transaction feature information time series data, and calculate the second prediction value o of the daily transaction feature information of the jth item at the current moment. 2j ;

[0034] The abnormal data determination module is used to obtain the first prediction value o 1j With the second predicted value o 2j Perform weighted processing to obtain the weighted prediction value o of the daily transaction feature information described in item j at the current moment j , the calculation formula is: j =ko 1j +(1-k)o 2j , where k is the weight coefficient of the predicted value, and k∈(0,1), and the prediction allowable error is set to d j , calculate the confidence interval of the predicted value of the daily transaction characteristic information described in item j [o jmin ,o jmax ], where o jmin , o jmax are the minimum and maximum values ​​of the confidence interval of the predicted value of the daily transaction characteristic information described in item j, respectively, and the calculation formula is: jmin =o j -d j , o jmax =o j +d j, according to the obtained confidence interval, determine the abnormal situation of the daily transaction characteristic information in the jth item at the current moment, wherein the determination principle is: if the sampling value of the daily transaction characteristic information data in the jth item at the current moment is within the confidence interval [o jmin ,o jmax ], it is judged as normal data; otherwise, it is judged as abnormal data;

[0035] The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0036] The present invention has the following beneficial effects:

[0037] Compared with the prior art, the method obtains the historical transaction time series data of the merchant to be supervised, extracts the daily transaction feature information in the transaction data, normalizes the acquired daily transaction feature information data, uses the normalized daily transaction feature information time series data as the input sequence of the LSTM neural network, and uses the neural network to obtain the first predicted value o of the daily transaction feature information of the jth item at the current moment. 1j , using the normalized daily transaction feature information time series data to build a statistical prediction model, and calculate the second prediction value o of the daily transaction feature information of the jth item at the current moment 2j , the first predicted value o will be obtained 1j With the second predicted value o 2j Perform weighted processing to obtain the weighted prediction value o of the daily transaction feature information described in item j at the current moment j , set the prediction tolerance to d j , calculate the confidence interval of the predicted value of the daily transaction characteristic information described in item j [o jmin ,o jmax ], determine the abnormal situation of the daily transaction characteristic information described in item j at the current moment according to the obtained confidence interval, and analyze the historical transaction data of the regulated merchants by using big data technology, so as to quickly screen and locate the abnormal points in the data, so as to timely identify abnormal or even illegal transaction behaviors from massive data, which is conducive to improving the management efficiency of retail supply chain data and promoting the smooth operation of the retail supply chain. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of a supply chain data management method based on big data of the present invention;

[0039] Figure 2 This is a structural block diagram of a supply chain data management system based on big data in the present invention;

[0040] Figure 3 Schematic diagram of the LSTM neural network structure constructed in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with specific implementation methods, wherein the accompanying drawings are only used for exemplary descriptions and represent only schematic diagrams rather than actual drawings, and should not be understood as limiting the present invention. In order to better illustrate the specific implementation methods of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product.

[0042] The specific implementation process of the technical solution of the present invention includes the following steps:

[0043] Step 1: Obtain the historical transaction time series data of the merchant to be regulated and extract the daily transaction feature information in the transaction data;

[0044] In this embodiment, a CNN neural network can be used to extract information features. Specifically, in view of the advantages of CNN in feature extraction and the good expression ability of activation functions in nonlinearity, as well as the good effects of many classic neural network models after the combination of CNN+LSTM, a multi-layer CNN network is added before the LSTM network to extract features, and the activation function is the ReLU function. Finally, after the LSTM layer, a Dense output layer and Softmax are added to complete the binary classification task. The specific network structure is as follows: Figure 3 As shown;

[0045] Step 2: Normalize the acquired daily transaction characteristic information data, where the daily transaction characteristic information includes daily transaction amount data, daily transaction number data, and daily transaction total volume data; the normalization formula is:

[0046]

[0047] In the formula, is the normalized value of the j-th daily transaction feature information data; x j is the sampling value of the j-th daily transaction characteristic information data; x jmin 、x jmax are the minimum and maximum values ​​in the historical transaction time series data of the j-th daily transaction feature information;

[0048] Step 3: Use the normalized daily transaction feature information time series data as the input sequence of the LSTM neural network, and use the neural network to obtain the first predicted value o of the j-th daily transaction feature information at the current moment 1j ;

[0049] Step 4: Use the normalized daily transaction feature information time series data to build a statistical prediction model and calculate the second prediction value o of the jth daily transaction feature information at the current moment 2j, where the expression of the statistical prediction model is:

[0050]

[0051] In the formula, It is represented as the normalized value of the t-th sampling value of the j-th daily transaction feature information; It is represented as the normalized value of the tpth sample value of the jth daily transaction feature information; is represented by the normalized value of the qth sample value of the jth daily transaction feature information; λ is a constant coefficient, and λ∈(0,1); m and n are both positive integers, and m+n=t; It is represented as the normalized value of the first sample value of the j-th daily transaction feature information;

[0052] Step 5: Obtain the first predicted value o 1j With the second predicted value o 2j Perform weighted processing to obtain the weighted prediction value o of the jth daily transaction feature information at the current moment j , the calculation formula is: j =ko 1j +(1-k)o 2j , where k is the weight coefficient of the predicted value, and k∈(0,1), the determination process includes the following steps:

[0053] Step 51: Obtain a subsequence of historical transaction time series data that is closest to the current moment and has a time length of T in the historical transaction time series data;

[0054] Step 52: Using the data in the subsequence as a verification sample, the first prediction value obtained by the LSTM neural network and the second prediction value obtained by the statistical prediction model are compared with the verification sample to obtain the classification result of the verification sample. The classification principle is:

[0055] Step 53: When the verification sample is normal data:

[0056] If the predicted value - validation sample value ≤ the prediction allowable error is d j , then it is classified as TP;

[0057] If the predicted value - validation sample value > the prediction allowable error is d j , then it is classified as FP;

[0058] When the validation sample is abnormal data:

[0059] If the predicted value - validation sample value ≤ the prediction allowable error is d j , then it is classified as FN;

[0060] If the predicted value - validation sample value > the prediction allowable error is dj , then it is classified as TN;

[0061] According to the classification results of the verification samples, the prediction accuracy A of the LSTM neural network and the statistical prediction model are calculated respectively. The calculation formula is: A = (TP + TN) / (TP + FP + FN + TN);

[0062] Step 54: Calculate the weight coefficient k of the prediction value with the obtained accuracy, and the calculation formula is: k = A1 / (A1+A2), where A1 and A2 are the prediction accuracy values ​​of the LSTM neural network and the statistical prediction model respectively obtained by using the historical transaction time series data subsequence data with a time length of T as the verification sample;

[0063] Step 6: Set the prediction tolerance to d j , calculate the confidence interval of the predicted value of the j-th daily transaction feature information [o jmin ,o jmax ], where o jmin , o jmax are the minimum and maximum values ​​of the confidence interval of the predicted value of the j-th daily transaction feature information, respectively. The calculation formula is: jmin =o j -d j , o jmax =o j +d j ;

[0064] Step 7: Determine the abnormality of the j-th daily transaction feature information at the current moment based on the obtained confidence interval, where the abnormality determination principle is:

[0065] If the sampling value of the jth daily transaction characteristic information data at the current moment is within the confidence interval [o jmin ,o jmax ], it is judged as normal data; otherwise, it is judged as abnormal data.

[0066] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A supply chain data management method based on big data, characterized in that: include: Obtaining historical transaction time series data of the merchant to be regulated, extracting daily transaction feature information from the transaction data, and normalizing the acquired daily transaction feature information data; The normalized daily transaction feature information time series data is used as the input sequence of the LSTM neural network, and the neural network is used to obtain the first predicted value o of the daily transaction feature information of the jth item at the current moment. 1j ; The statistical prediction model is constructed using the normalized daily transaction feature information time series data, and the second prediction value o of the daily transaction feature information of the jth item at the current moment is calculated. 2j , where the expression of the statistical prediction model is: In the formula, It is represented as the normalized value of the t-th sampling value of the daily transaction characteristic information described in the j-th item; It is represented as the normalized value of the tp-th sampling value of the daily transaction feature information described in the j-th item; is represented by the normalized value of the qth sample value of the daily transaction feature information described in the jth item; λ is a constant coefficient, and λ∈(0,1); m and n are both positive integers, and m+n=t; It is represented as the normalized value of the first sample value of the daily transaction characteristic information described in the j-th item; The first predicted value o will be obtained 1j With the second predicted value o 2j Perform weighted processing to obtain the weighted prediction value o of the daily transaction feature information described in item j at the current moment j , the calculation formula is: j =ko 1j +(1-k)o 2j , where k is the weight coefficient of the predicted value, and k∈(0,1); Set the prediction tolerance to d j , calculate the confidence interval of the predicted value of the daily transaction characteristic information described in item j Among them, jmin , o jmax are the minimum and maximum values ​​of the confidence interval of the predicted value of the daily transaction characteristic information described in item j, respectively, and the calculation formula is: jmin =o j -d j , o jmax =o j +d j ; The abnormality of the daily transaction characteristic information described in the j-th item at the current moment is determined based on the obtained confidence interval.

2. The supply chain data management method based on big data according to claim 1, characterized in that: The normalization formula is: In the formula, is the normalized value of the daily transaction characteristic information data described in item j; j is the sampling value of the daily transaction characteristic information data described in item j; jmin 、x jmax are respectively the minimum and maximum values ​​in the historical transaction time series data of the daily transaction feature information described in item j.

3. The supply chain data management method based on big data according to claim 1, characterized in that: The daily transaction characteristic information includes daily transaction amount data, daily transaction number data, and daily transaction volume data.

4. The supply chain data management method based on big data according to claim 1, characterized in that: The principle for determining abnormal situations of the daily transaction characteristic information described in item j at the current moment is: If the daily transaction characteristic information data sampling value described in item j at the current moment is within the confidence interval If it is within, it is judged as normal data; Otherwise, it is determined to be abnormal data.

5. The supply chain data management method based on big data according to claim 1, characterized in that: The process of determining the weight coefficient k of the predicted value includes the following steps: Obtain the historical transaction time series data subsequence that is closest to the current time and has a time length of T in the historical transaction time series data; The data in the subsequence is used as the validation sample. The first prediction value obtained by the LSTM neural network and the second prediction value obtained by the statistical prediction model are compared with the validation sample to obtain the classification result of the validation sample. The classification principle is: When the validation sample is normal data: If the predicted value - validation sample value ≤ the prediction allowable error is d j , then it is classified as TP; If the predicted value - validation sample value > the prediction allowable error is d j , then it is classified as FP; When the validation sample is abnormal data: If the predicted value - validation sample value ≤ the prediction allowable error is d j , then it is classified as FN; If the predicted value - validation sample value > the prediction allowable error is d j , then it is classified as TN; According to the classification results of the verification samples, the prediction accuracy A of the LSTM neural network and the statistical prediction model are calculated respectively. The calculation formula is: A = (TP + TN) / (TP + FP + FN + TN); The weight coefficient k of the predicted value is calculated with the obtained accuracy, and the calculation formula is: k=A1 / (A1+A2), where A1 and A2 are the prediction accuracy values ​​of the LSTM neural network and the statistical prediction model respectively obtained by using the subsequence data of the historical transaction time series data with a time length of T as the verification sample.

6. A supply chain data management system based on big data, characterized in that: It includes data acquisition module, feature information extraction module, data processing module, neural network module, data statistics prediction module and abnormal data determination module; The data acquisition module is used to acquire the historical transaction time series data of the merchant to be supervised; The feature information extraction module is used to extract daily transaction feature information from the transaction data, wherein the daily transaction feature information includes daily transaction amount data, daily transaction number data, and daily transaction total volume data; The data processing module is used to normalize the acquired daily transaction characteristic information data, wherein the data normalization processing formula is: In the formula, x j is the normalized value of the daily transaction characteristic information data described in item j; j is the sampling value of the daily transaction characteristic information data described in item j; jmin 、x jmax are respectively the minimum and maximum values ​​in the historical transaction time series data of the daily transaction characteristic information described in item j; The neural network module is used to construct an LSTM neural network, and use the neural network to obtain the first predicted value o of the daily transaction feature information of the jth item at the current moment. 1j , wherein the LSTM neural network uses the normalized daily transaction feature information time series data as an input sequence; The data statistical prediction module is used to construct a statistical prediction model using the normalized daily transaction feature information time series data, and calculate the second prediction value o of the daily transaction feature information of the jth item at the current moment. 2j , where the expression of the statistical prediction model is: In the formula, It is represented as the normalized value of the t-th sampling value of the daily transaction characteristic information described in the j-th item; It is represented as the normalized value of the tp-th sampling value of the daily transaction feature information described in the j-th item; is represented by the normalized value of the qth sample value of the daily transaction feature information described in the jth item; λ is a constant coefficient, and λ∈(0,1); m and n are both positive integers, and m+n=t; It is represented as the normalized value of the first sample value of the daily transaction characteristic information described in the j-th item; The abnormal data determination module is used to obtain the first prediction value o 1j With the second predicted value o 2j Perform weighted processing to obtain the weighted prediction value o of the daily transaction feature information described in item j at the current moment j , the calculation formula is: j =ko 1j +(1-k)o 2j , where k is the weight coefficient of the predicted value, and k∈(0,1), and the prediction allowable error is set to d j , calculate the confidence interval of the predicted value of the daily transaction characteristic information described in item j Among them, jmin , o jmax are the minimum and maximum values ​​of the confidence interval of the predicted value of the daily transaction characteristic information described in item j, respectively, and the calculation formula is: jmin =o j -d j , o jmax =o j +d j , based on the obtained confidence interval, determine the abnormality of the daily transaction characteristic information described in the jth item at the current moment, where the determination principle is: If the daily transaction characteristic information data sampling value described in item j at the current moment is within the confidence interval If it is within the range, it is judged as normal data; otherwise, it is judged as abnormal data.

7. A supply chain data management system based on big data according to claim 6, characterized in that: The system comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the method according to any one of claims 1 to 5 are implemented when the processor executes the program.

Images