Method for studying and judging multi-dimensional features of business object through big data
Through cleaning, decomposition and multi-dimensional analysis of business object data, combined with logical prediction models, the inconvenience and inaccurate problems of big data processing and analysis in the existing technology are solved, and more accurate business decision-making and operation optimization are achieved.
Patent Information
- Application Number
- CN202411205595.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-22
AI Technical Summary
The existing technology is difficult to effectively process and analyze big data, resulting in inconvenience and inaccuracy in decision-making, especially in the understanding and prediction of multi-dimensional characteristics of business objects.
By collecting data related to business objects, perform data cleaning, preprocessing, decomposition, feature extraction and in-depth analysis, combining multi-dimensional analysis and logical prediction models, predict future data change trends and formulate strategies.
It achieves data accuracy, completeness and consistency, improves the accuracy and efficiency of business decisions, helps enterprises more accurately evaluate potential and value, optimize operational strategies, reduce inventory backlogs and reduce operating costs.
Smart Images

Figure CN119005808A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and more specifically, to a method for analyzing multi-dimensional features of business objects using big data. Background Art
[0002] As technologies such as the Internet of Things, social networks, and cloud computing continue to integrate into our lives and the existing computing power, storage space, and network bandwidth develop rapidly, the data accumulated by humans continues to grow in many fields such as the Internet, communications, finance, commerce, and medical care. Huge amounts of data are generated every day in all walks of life. By mining and analyzing the data, valuable information hidden in the massive data can be extracted, and then the information can be applied to real business, improving production efficiency, optimizing management, etc.
[0003] To complete an overall business and realize the transformation of data from information to knowledge, a complete system is needed to manage these individual functions, form a business process, and finally complete the data mining and analysis functions, turning information into valuable conclusions and assisting relevant personnel in making decisions.
[0004] Therefore, in order to improve the convenience and accuracy of decision-making of relevant personnel, a method for using big data to analyze the multi-dimensional characteristics of business objects is proposed. Summary of the invention
[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a method for analyzing multi-dimensional features of business objects using big data to solve the problems raised in the above-mentioned background technology.
[0006] To achieve the above object, the present invention provides the following technical solution: a method for analyzing multi-dimensional features of business objects using big data, comprising the following steps: S1: First, collect all data related to the business object, including historical data, real-time data, user behavior data, transaction data, etc. S2: After collecting the data, data cleaning and preprocessing are performed to ensure the accuracy and consistency of the data, including removing duplicate data, processing missing values, correcting erroneous data, etc.; S3: Decompose the collected data according to different dimensions, including time, geographic location, user type, product type, etc. Through dimensional decomposition, we can have a deeper understanding of various aspects of the business object; S4: Extract key features related to the business object in each dimension. These features include statistical indicators such as mean and standard deviation, as well as user behavior patterns and purchasing habits. S5: Conduct in-depth analysis on the extracted features to understand the multi-dimensional characteristics of the business object, including comparative analysis, grouping analysis, regression analysis, indicator analysis, etc. Through these analyses, the essential characteristics and development laws of the business object are revealed; S6: Based on the extracted features and analysis results, judge and predict the future data change trend to discover potential risks and opportunities in advance and formulate corresponding strategies; S7: Visualize the analysis results in the form of charts, images, etc. to more intuitively understand the multi-dimensional characteristics of the business object.
[0007] Preferably, in S2, firstly, anomaly detection processing is performed on the collected data, then missing value filling is performed on the data after anomaly detection and processing, and finally, the data after missing value filling is standardized.
[0008] Preferably, in S2, when performing anomaly detection processing on data, the specific formula is: Outlier judgment conditions: Less than Values greater than are considered outliers because they are well below the “normal” range for the dataset; The values of are also considered outliers because these values are far above the "normal" range of the data set, and the detected outliers are deleted; in, is the interquartile range, and the formula is specifically defined as: the third quartile ( , i.e. 75% quantile) and the first quartile ( , i.e. the 25% quantile), is used to measure the dispersion of the data; The first quartile is defined as the value at the 25% position after sorting the data from small to large. It means that 25% of the data in the data set is less than or equal to ; The third quartile is defined as the value at the 75% position after sorting the data from small to large. It means that 75% of the data in the data set is less than or equal to .
[0009] Preferably, in S2, when filling missing values in the data after anomaly detection and processing, the specific formula is: in, Represents the mean of the data, which is used to fill in missing values.
[0010] Preferably, in S2, the data after missing value filling is standardized, and the specific formula and steps are: First, set the number of padding data items to item; Then, the standard deviation of the data is calculated based on the filled data. The specific formula is: in, represents the standard deviation of the data, Indicates the number of data items, Indicates The data value of the item; Finally, according to the calculated mean and standard deviation , the total data is standardized, and the specific processing formula is: in, Indicates The standard value of the item.
[0011] Preferably, in S3, the standardized data is grouped according to different dimensions, so as to understand the specific conditions of various aspects of the business object. The specific grouping steps are as follows: Step 1: First, determine a K value, that is, the number of clusters that you want to obtain by clustering the data set. The K value is determined by the number of dimensional features of the business object, and randomly select K data points in the data set as the initial centroid; Step 2: For each data point in the data set, calculate its distance to each centroid and assign the data point to the cluster represented by the centroid closest to it; Step 3: For each cluster assigned above, calculate the average value of all data points in the cluster and set this average value as the new centroid; Step 4: Repeat steps 2 and 3 above until the centroid no longer changes significantly. Step 5: When the algorithm stops, each data point is assigned to a cluster, forming k non-overlapping groups. Each group is represented by a centroid, and the distance between the data points within the group and the centroid is relatively small, while the distance between the centroids of different groups is relatively large, thus completing the data grouping of different dimensional features.
[0012] Preferably, in step 2, the distance calculation formula between each data point and each centroid in the data set is specifically: in, Indicates data points; Indicates centroid; The dimension representing the data points and centroids, i.e. the number of features; Represents data points The kth eigenvalue of ; Represents the centroid The kth eigenvalue of ; This formula calculates the Euclidean distance between two points by calculating the sum of the squares of the differences between the data point and the centroid in each dimension and then taking the square root.
[0013] Preferably, in step 3, the calculation formula for updating the centroid is: in, represents the updated centroid; Indicates clusters, that is, all the clusters assigned to A set of data points that are clustered; Representation clustering The number of data points in , i.e., the size of the cluster; Representation clustering A data point in Represents clustering All data points in The sum of .
[0014] Preferably, after separating the feature data of each dimension in S4, the data of each dimension is processed separately, and at the same time, comparative analysis is performed on the features of different dimensions.
[0015] Preferably, in S6, the future data change trend is judged and predicted, and a prediction model is established. Assuming that the given input feature is X and the output variable Y is 1, the prediction probability of the feature is calculated, and X contains n features, i.e., dimensions, and each feature is expressed as ( ), the specific algorithm formula is: in, It indicates the probability that Y takes the value of 1 given X; is the coefficient of the logistic prediction model, The intercept term, Is with characteristics The corresponding coefficient.
[0016] The method for analyzing multi-dimensional features of business objects using big data provided by the present invention has the following beneficial effects: The present invention performs abnormal detection and processing on all the collected data related to the business object to ensure the accuracy of the data, and fills the missing values of the processed data to ensure the integrity of the data, and performs standardization processing on the complete data after filling, so as to ensure the consistency of the data, so as to facilitate the subsequent query and analysis of the data, and at the same time, the standardized data is accurately grouped according to different dimensional features, and multiple iterations are performed during the grouping process to ensure the accuracy of the data grouping, so as to ensure that each group of data reflects a dimensional feature, so as to facilitate the visual comparison of multi-dimensional features, so as to facilitate the analysis of each dimensional feature, and to facilitate the judgment of future data change trends, so as to estimate risks in advance, and enterprises can more accurately evaluate the potential and value of different business objects, which is conducive to the formulation of future development direction strategies by enterprises, so as to establish a logical prediction model according to the standardized data, and iterate and update the dimensional coefficients in the logical prediction model in real time, so as to facilitate enterprises to more accurately predict market trends and business development directions, so as to adjust the operation strategy in advance. This helps enterprises reduce inventory backlogs, reduce operating costs, and improve operating efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a schematic diagram of the method flow of the present invention.
[0018] Figure 2 It is a schematic diagram of the process of grouping the standardized data according to different dimensions according to the present invention. DETAILED DESCRIPTION
[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0020] As attached Figure 1 A method for analyzing multi-dimensional features of business objects with big data is shown, comprising the following steps: S1: First, collect all data related to the business object, including historical data, real-time data, user behavior data, transaction data, etc. S2: After collecting the data, data cleaning and preprocessing are performed to ensure the accuracy and consistency of the data, including removing duplicate data, processing missing values, correcting erroneous data, etc.; When performing anomaly detection on data, the specific formula is: Outlier judgment conditions: Less than Values greater than are considered outliers because they are well below the “normal” range for the dataset; The values of are also considered outliers because these values are far above the "normal" range of the data set, and the detected outliers are deleted; in, is the interquartile range, and the formula is specifically defined as: the third quartile ( , i.e. 75% quantile) and the first quartile ( , i.e. the 25% quantile), is used to measure the dispersion of the data; The first quartile is defined as the value at the 25% position after sorting the data from small to large. It means that 25% of the data in the data set is less than or equal to ; The third quartile is defined as the value at the 75% position after sorting the data from small to large. It means that 75% of the data in the data set is less than or equal to .
[0021] When filling missing values for data after anomaly detection and processing, the specific formula is: in, Represents the mean of the data, which is used to fill in missing values.
[0022] By calculating the mean Fill in the missing position. The specific program running code is expressed as: # Assume df is a pandas DataFrame with columns named 'column_name' mean_value = df['column_name'].mean() df['column_name'].fillna(mean_value, inplace=True).
[0023] The data after missing value filling is standardized. The specific formula and steps are as follows: First, set the number of padding data items to item; Then, the standard deviation of the data is calculated based on the filled data. The specific formula is: in, represents the standard deviation of the data, Indicates the number of data items, Indicates The data value of the item.
[0024] Specifically, by filling After the data items are Items, calculate the standard deviation of the total item data, thereby reducing the standard error of the total data and improving the accuracy of the data; Finally, according to the calculated mean and standard deviation , the total data is standardized, and the specific processing formula is: in, Indicates The standard value of the item.
[0025] Specifically, by standardizing the total data, the data format and structure become more unified, which facilitates subsequent query and analysis of the data, allowing for more efficient discovery of valuable information hidden in the data and improved decision-making accuracy and efficiency. Through standardized processing, big data analysis services can obtain more accurate and comprehensive data information, thereby supporting the optimization of business decisions. Enterprises can formulate more scientific and reasonable business strategies based on standardized data.
[0026] S3: Decompose the collected data according to different dimensions, including time, geographic location, user type, product type, etc. Through dimensional decomposition, we can have a deeper understanding of various aspects of the business object; like Figure 2 As shown, the specific grouping steps are as follows: Step 1: First, determine a K value, that is, the number of clusters that you want to obtain by clustering the data set. The K value is determined by the number of dimensional features of the business object, and randomly select K data points in the data set as the initial centroid; Step 2: For each data point in the data set, calculate its distance to each centroid and assign the data point to the cluster represented by the centroid closest to it; The distance calculation formula for each data point and each centroid in the data set is as follows: in, Indicates data points; Indicates centroid; The dimension representing the data points and centroids, i.e. the number of features; Represents data points The kth eigenvalue of ; Represents the centroid The kth eigenvalue of ; This formula calculates the Euclidean distance between two points by calculating the sum of the squares of the differences between the data point and the centroid in each dimension and then taking the square root; Step 3: For each cluster assigned above, calculate the average value of all data points in the cluster and set this average value as the new centroid; The updated calculation formula for the centroid is: in, represents the updated centroid; Indicates clusters, that is, all the clusters assigned to A set of data points that are clustered; Representation clustering The number of data points in , i.e., the size of the cluster; Representation clustering A data point in Represents clustering All data points in The sum of Step 4: Repeat steps 2 and 3 above until the centroid no longer changes significantly. Step 5: When the algorithm stops, each data point is assigned to a cluster, forming k mutually exclusive groups. Each group is represented by a centroid, and the distance between the data points in a group and the centroid is relatively small, while the distance between the centroids of different groups is relatively large, thus completing the data grouping of different dimensional features; S4: Extract key features related to the business object in each dimension. These features include statistical indicators such as mean and standard deviation, as well as user behavior patterns and purchasing habits. S5: Conduct in-depth analysis on the extracted features to understand the multi-dimensional characteristics of the business object, including comparative analysis, grouping analysis, regression analysis, indicator analysis, etc. Through these analyses, the essential characteristics and development laws of the business object are revealed; S6: Based on the extracted features and analysis results, judge and predict the future data change trend to discover potential risks and opportunities in advance and formulate corresponding strategies; To judge and predict the future data change trend and establish a prediction model, let the input feature be X, the output variable Y be 1, calculate the prediction probability of the feature, and X contains n features, i.e., dimensions, and each feature is expressed as ( ), the specific algorithm formula is: in, It indicates the probability that Y takes the value of 1 given X; is the coefficient of the logistic prediction model, The intercept term, Is with characteristics The corresponding coefficients; Specifically, the coefficients of the logistic prediction model are The maximum likelihood estimation method is used for determination, and the specific method is as follows: Suppose there are m training samples, each with n dimensional features and a label ,right samples, whose labels are , the eigenvector is .
[0027] Then the logistic prediction model predicts that the probability of the sample label being 1 is: in, is the output of the linear predictor.
[0028] Therefore, the The log-likelihood of a sample is: Will Substitute the expression into the above formula and get: = Therefore, the log-likelihood function for the entire data set is: In order to find The largest Value, need to be calculated About each parameter The partial derivative of because ,so (When j=0, =1, indicating the intercept term).
[0029] Substituting into the above formula, we get: Then the gradient expression is obtained, and the parameter To update: in, is the learning rate, which controls the step size of parameter updates.
[0030] The following is pseudo code for training a logistic prediction model using the gradient ascent algorithm: Parameters = ) is a small random value; Setting the learning rate and the number of iterations max_iter; For each iteration t=1 to max_iter; Calculate the linear prediction value for each sample ; Calculate the predicted probability for each sample ; Compute the gradient: Update Parameters Return Parameter .
[0031] By calculating the coefficient of each dimension as above, when judging the business object, directly input the dimension feature X corresponding to the business object, and you can use Calculate the probability of the corresponding feature of the business object, so that business personnel can judge and predict the future data change trend, which helps to discover potential risks and opportunities in advance and formulate corresponding strategies S7: Visualize the analysis results in the form of charts, images, etc. to more intuitively understand the multi-dimensional characteristics of the business object.
[0032] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for analyzing the multi-dimensional characteristics of business objects using big data, characterized in that: The following steps are involved: S1: First, collect all data related to the business object, including historical data, real-time data, user behavior data, and transaction data; S2: After collecting the data, data cleaning and preprocessing are performed to ensure the accuracy and consistency of the data, including removing duplicate data, processing missing values, and correcting erroneous data; S3: Decompose the collected data according to different dimensions, including time, geographic location, user type, product type, etc. Through dimensional decomposition, we can have a deeper understanding of various aspects of the business object; S4: Extract key features related to the business object in each dimension, including average and standard deviation statistical indicators, as well as user behavior patterns and purchasing habits; S5: Conduct in-depth analysis on the extracted features to understand the multi-dimensional characteristics of the business object, including comparative analysis, grouping analysis, regression analysis, indicator analysis, etc. Through these analyses, the essential characteristics and development laws of the business object are revealed; S6: Based on the extracted features and analysis results, judge and predict the future data change trend to discover potential risks and opportunities in advance and formulate corresponding strategies; S7: Visualize the analysis results in the form of charts, images, etc. to more intuitively understand the multi-dimensional characteristics of the business object.
2. According to claim 1, a method for analyzing multi-dimensional features of business objects using big data is characterized in that: In S2, firstly, anomaly detection processing is performed on the collected data, then missing value filling is performed on the data after anomaly detection and processing, and finally, the data after missing value filling is standardized.
3. According to claim 2, a method for analyzing multi-dimensional features of business objects using big data is characterized in that: In S2, when performing anomaly detection processing on data, the specific formula is: Outlier judgment conditions: Less than Values greater than are considered outliers because they are well below the "normal" range of the dataset; The values of are also considered outliers because they are far above the "normal" range of the data set, and the detected outliers are deleted; in, is the interquartile range, and the formula is specifically defined as: the third quartile ( , i.e. 75% quantile) and the first quartile ( , i.e. the 25% quantile), is used to measure the dispersion of the data; The first quartile is defined as the value at the 25% position after sorting the data from small to large. It means that 25% of the data in the data set is less than or equal to ; The third quartile is defined as the value at the 75% position after sorting the data from small to large. It means that 75% of the data in the data set is less than or equal to .
4. According to claim 3, a method for analyzing multi-dimensional features of business objects using big data is characterized in that: In S2, when filling missing values for the data after anomaly detection and processing, the specific formula is: in, Represents the mean of the data, which is used to fill in missing values.
5. According to claim 4, a method for analyzing multi-dimensional features of business objects using big data is characterized in that: In S2, the data after missing value filling is standardized, and the specific formula and steps are as follows: First, set the number of padding data items to item; Then, the standard deviation of the data is calculated based on the filled data. The specific formula is: in, represents the standard deviation of the data, Indicates the number of data items, Indicates The data value of the item; Finally, according to the calculated mean and standard deviation , the total data is standardized, and the specific processing formula is: in, Indicates The standard value of the item.
6. According to claim 5, a method for analyzing multi-dimensional features of business objects using big data is characterized in that: In S3, the standardized data is grouped according to different dimensions, so as to understand the specific conditions of various aspects of the business object. The specific grouping steps are as follows: Step 1: First, determine a K value, that is, the number of clusters that you want to obtain by clustering the data set. The K value is determined by the number of dimensional features of the business object, and randomly select K data points in the data set as the initial centroid; Step 2: For each data point in the data set, calculate its distance to each centroid and assign the data point to the cluster represented by the centroid closest to it; Step 3: For each cluster assigned above, calculate the average value of all data points in the cluster and set this average value as the new centroid; Step 4: Repeat steps 2 and 3 above until the centroid no longer changes significantly. Step 5: When the algorithm stops, each data point is assigned to a cluster, forming k non-overlapping groups. Each group is represented by a centroid, and the distance between the data points within the group and the centroid is relatively small, while the distance between the centroids of different groups is relatively large, thus completing the data grouping of different dimensional features.
7. According to claim 6, a method for analyzing multi-dimensional features of business objects using big data is characterized in that: In step 2, the distance calculation formula between each data point and each centroid in the data set is specifically: in, Indicates data points; Indicates centroid; The dimension representing the data points and centroids, i.e. the number of features; Represents data points The kth eigenvalue of ; Represents the centroid The kth eigenvalue of ; This formula calculates the Euclidean distance between two points by calculating the sum of the squares of the differences between the data point and the centroid in each dimension and then taking the square root.
8. According to claim 7, a method for analyzing multi-dimensional features of business objects using big data is characterized in that: In step 3, the calculation formula for updating the centroid is: in, represents the updated centroid; Indicates clusters, that is, all the clusters assigned to A set of data points that are clustered; Representation clustering The number of data points in , i.e., the size of the cluster; Representation clustering A data point in Represents clustering All data points in The sum of .
9. The method for analyzing multi-dimensional features of business objects using big data according to claim 8 is characterized in that: After separating the feature data of each dimension in S4, the data of each dimension is processed separately, and at the same time, comparative analysis is performed on the features of different dimensions.
10. The method for analyzing multi-dimensional features of business objects using big data according to claim 9 is characterized in that: In S6, the future data change trend is judged and predicted, and a prediction model is established. Assuming that the input feature is X and the output variable Y is 1, the prediction probability of the feature is calculated, and X contains n features, i.e., dimensions, and each feature is expressed as ( ), the specific algorithm formula is: in, It indicates the probability that Y takes the value of 1 given X; is the coefficient of the logistic prediction model, The intercept term, Is with characteristics The corresponding coefficient.
