Portrayal distribution construction method of numerical value type label and related equipment thereof

Abstract

The invention discloses a figure distribution construction method of numerical value type tags and related equipment thereof, and belongs to the technical field of big data. The method comprises the following steps: obtaining total data used for constructing portrait distribution, the total data being numerical value type data; using a kernel density estimation algorithm to calculate the density distribution condition of the total data; performing data binning on the total data based on the density distribution condition to obtain a plurality of data intervals; clustering analysis is conducted on the data intervals, a plurality of data groups are obtained, and the number of the data groups is smaller than or equal to the number of the data intervals; and performing data visualization conversion on the plurality of data groups by using a preset visualization tool to generate data portrait distribution. The invention further relates to the technical field of block chains, and full data is stored in a block chain network. Visual figure distribution of numerical value type labels can be generated, a user can check and analyze the figure distribution efficiently, and the problem of figure distribution construction for processing high-cardinal numerical value data is solved.

Description

Technical Field

[0001] The present application belongs to the field of big data technology, and specifically relates to a method for constructing a portrait distribution of numerical type labels and related equipment. Background Art

[0002] The portrait of numerical type tags refers to the data description and analysis of tags with numerical characteristics to form a specific portrait of the data object (such as users, products, etc.) under the tag. The characteristics of this type of tag are that the values ​​are continuous or relatively continuous numerical values, which can reflect the specific performance or characteristics of the data object in a certain dimension.

[0003] Taking the consumption capacity distribution of e-commerce platforms as an example, if you want to understand the consumption capacity distribution of users on the platform in order to better conduct user profiling and personalized recommendations, then in this scenario, "consumption capacity" is a numerical type label, and its value can be the amount of consumption by the user in a certain period of time. By generating specific portraits for user groups with different consumption capacities, personalized recommendations and marketing strategies can be formulated accordingly. For example, for low-spending groups, cost-effective products and promotions can be recommended; for high-income groups, high-quality, high-value-added products and services can be recommended.

[0004] However, the user's consumption amount can be arbitrarily subdivided within its value range, and there are countless possible values ​​between each segment, resulting in its value cardinality (that is, the number of different values) often being high. This makes it difficult to directly view or analyze all possible values ​​when constructing a user's consumption amount profile distribution, and also increases the complexity of data processing. Summary of the invention

[0005] The purpose of the embodiments of the present application is to propose a method, device, computer equipment and storage medium for constructing a portrait distribution of numerical type tags, so as to solve the technical problem that the existing numerical type tags have a high value base, which makes it difficult to directly view or analyze all possible values ​​when constructing a user's consumption amount portrait distribution, and also increases the complexity of data processing.

[0006] In order to solve the above technical problems, the embodiment of the present application provides a method for constructing a portrait distribution of a numerical type label, which adopts the following technical solution:

[0007] A method for constructing a profile distribution of a numerical type label, comprising:

[0008] Obtain the full amount of data used to construct the profile distribution, where the full amount of data is numerical type data;

[0009] Use the kernel density estimation algorithm to calculate the density distribution of the full data;

[0010] Based on the density distribution, the full amount of data is binned to obtain several data intervals;

[0011] Performing cluster analysis on a number of data intervals to obtain a number of data groups, wherein the number of data groups is less than or equal to the number of data intervals;

[0012] Use preset visualization tools to perform data visualization transformation on several data groups and generate data portrait distribution.

[0013] Furthermore, after the step of obtaining the full amount of data for constructing the profile distribution, it also includes:

[0014] Perform data anomaly detection on the entire data to determine the abnormal data values ​​in the entire data;

[0015] Perform data cleaning on all data based on abnormal data values;

[0016] The full amount of data that has completed data cleaning is subjected to data standardization processing to obtain standardized full amount of data.

[0017] Furthermore, the step of using the kernel density estimation algorithm to calculate the density distribution of the entire data specifically includes:

[0018] Select a kernel function that matches the full data set and define the range of the initial bandwidth.

[0019] Within the range of the initial bandwidth, the cross-validation method is used to optimize the bandwidth parameters to obtain the optimal bandwidth parameters;

[0020] The optimal bandwidth parameter is used as the bandwidth value of the kernel density calculation, and the density distribution curve of the full data is drawn using the preset visual density curve drawing tool combined with the kernel function;

[0021] Get the density distribution of the full data from the density distribution curve.

[0022] Furthermore, the formula for bandwidth parameter optimization is as follows:

[0023]

[0024] In the formula, σ is the standard deviation of the full data, and n is the number of samples.

[0025] Furthermore, the steps of binning the full amount of data based on the density distribution to obtain several data intervals specifically include:

[0026] Determine the peak and valley points of the density distribution curve according to the density distribution of the full amount of data;

[0027] Determine the inflection point of density change according to the peak and valley points of the density distribution curve;

[0028] Set the data binning interval according to the inflection point of density change;

[0029] The full amount of data is binned according to the data binning interval to obtain several data intervals.

[0030] Furthermore, the steps of performing cluster analysis on a plurality of data intervals to obtain a plurality of data groups specifically include:

[0031] Perform feature analysis on each data interval to obtain interval data features of each data interval;

[0032] Calculate the data feature similarity between each data interval based on the interval data feature;

[0033] Cluster analysis is performed on several data intervals according to the similarity of data features to obtain several data groups.

[0034] Furthermore, after performing cluster analysis on a plurality of data intervals to obtain a plurality of data groups, the method further includes:

[0035] Storing several data groups in preset application configuration libraries respectively;

[0036] The steps of using preset visualization tools to perform data visualization transformation on several data groups and generate data portrait distribution include:

[0037] Receive portrait building instructions and determine the data viewing permissions of the initiator of the instructions;

[0038] Read the data group that matches the data viewing permission in the application configuration library to obtain the target data group;

[0039] Use visualization tools to perform data visualization transformation on the target data group and generate data portrait distribution.

[0040] In order to solve the above technical problems, the embodiment of the present application further provides a device for constructing a distribution image of a numerical type label, which adopts the following technical solution:

[0041] A device for constructing a distribution image of a numerical type label, comprising:

[0042] A data acquisition module is used to acquire the full amount of data used to construct the profile distribution, wherein the full amount of data is numerical type data;

[0043] Density calculation module, used to calculate the density distribution of the entire data using the kernel density estimation algorithm;

[0044] The data binning module is used to bin the full amount of data based on density distribution to obtain several data intervals;

[0045] A cluster analysis module, used to perform cluster analysis on a number of data intervals to obtain a number of data groups, wherein the number of data groups is less than or equal to the number of data intervals;

[0046] The portrait distribution module is used to use preset visualization tools to perform data visualization transformation on several data groups and generate data portrait distribution.

[0047] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0048] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for constructing a portrait distribution of a numerical type label as described in any one of the above items are implemented.

[0049] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0050] A computer-readable storage medium having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by a processor, implement the steps of a method for constructing a portrait distribution of a numerical type label as described in any one of the above.

[0051] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0052] The present application discloses a method for constructing a portrait distribution of a numerical type label and its related equipment, belonging to the field of big data technology. The present application obtains the full amount of data used to construct the portrait distribution, wherein the full amount of data is numerical type data; uses the kernel density estimation algorithm to calculate the density distribution of the full amount of data; based on the density distribution, the full amount of data is binned to obtain several data intervals; cluster analysis is performed on several data intervals to obtain several data groups, wherein the number of data groups is less than or equal to the number of data intervals; and a preset visualization tool is used to perform data visualization transformation on several data groups to generate a data portrait distribution. The present application estimates the distribution smoothing of numerical data through the kernel density algorithm to avoid directly processing the complexity of data with high cardinality and continuous values, and then the binning strategy based on the density distribution effectively reduces the number of numerical intervals, making complex data more concise, and then combines cluster analysis to merge similar intervals into data groups, further simplifying the data dimension, and finally generates an intuitive portrait distribution of numerical type labels through a visualization tool, so that users can efficiently view and analyze, solving the problem of constructing a portrait distribution for processing high cardinality numerical data. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 An exemplary system architecture diagram is shown in which the present application can be applied;

[0055] Figure 2 A flowchart of an embodiment of a method for constructing a distribution of a numerical type label according to the present application is shown;

[0056] Figure 3 A schematic diagram of the structure of an embodiment of a device for constructing a distribution image of a numerical type tag according to the present application is shown;

[0057] Figure 4 A schematic structural diagram of an embodiment of a computer device according to the present application is shown. DETAILED DESCRIPTION

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0059] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0060] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0061] like Figure 1As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0062] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0063] The terminal device 101 may be any electronic device having a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 may also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III, Moving ...

[0064] Video Experts Compression Standard Audio Layer IV) players, laptops and desktop computers, etc.

[0065] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0066] It should be noted that the method for constructing a distribution portrait of a numerical type tag provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the apparatus for constructing a distribution portrait of a numerical type tag is generally arranged in a server / terminal device.

[0067] It should be understood that Figure 1 The numbers of terminal devices, networks and servers in the embodiment are merely illustrative. The above system may have any number of terminal devices, networks and servers according to implementation requirements.

[0068] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for constructing a distribution image of a numerical type tag according to the present application. The method for constructing a distribution image of a numerical type tag comprises the following steps:

[0069] S201, obtaining full data for constructing a portrait distribution, wherein the full data is numerical type data.

[0070] Specifically, the system first obtains the full amount of data used to build the profile distribution. The data type obtained is numerical data, which usually has the characteristics of continuity and high cardinality. Numerical tags include consumption amount, age, income, rating, etc. Since these data have a large range and many value conditions, directly processing these raw data will bring complexity in storage and analysis.

[0071] To solve this problem, we must first ensure that the collection of full data is comprehensive and accurate, covering the characteristics of all users or objects, so that there is sufficient data support for subsequent processing and analysis steps. In actual applications, data can be obtained from different systems or databases, and may involve user behavior data, consumption data, or other business data.

[0072] It should be noted that basic data cleaning and preprocessing are also required at this stage to ensure the quality of the data. The processing involves removing missing values, outliers, etc. to ensure that the data processed in subsequent steps is complete and meaningful.

[0073] S202, using a kernel density estimation algorithm to calculate the density distribution of the entire data.

[0074] Specifically, kernel density estimation (KDE) is a parameter-free estimation technique used to infer the probability density distribution of numerical data. Compared with the traditional histogram method, KDE provides a smooth distribution curve and avoids the distortion problem caused by improper interval division of the histogram. The core idea of ​​the algorithm is to smooth each data point by using a kernel function (such as a Gaussian kernel), and then summarize all the data to form a density estimate of the data.

[0075] When calculating, bandwidth (or smoothing parameter) is a key factor. The bandwidth determines the width of the kernel function. When the bandwidth is small, the estimated density distribution will be very detailed, but it may overfit the noise in the data; when the bandwidth is large, the estimated result is smoother, but it may ignore the details in the data.

[0076] In actual operation, kernel density estimation can identify the peaks and valleys of the data, revealing the range of values ​​within which the data has a high concentration, and provide a basis for the next step of data binning. Through the KDE algorithm, the distribution characteristics of the full data are more clearly displayed, making subsequent analysis more accurate.

[0077] S203, binning the entire data based on density distribution to obtain a number of data intervals.

[0078] Specifically, data binning is to divide continuous numerical data into multiple intervals in order to simplify the complexity of the data and make it easier to process and analyze the data later.

[0079] Binning based on density distribution means determining the specific data binning operation based on the actual distribution of the data, usually using the density inflection point in the density estimation as the dividing line. In the high-density part, the data is relatively concentrated, so the binning can be refined to capture more details; while in the low-density part, the data is relatively sparse, and a wider binning range can be used to reduce unnecessary complexity.

[0080] In the binning process, combined with the distribution curve of kernel density estimation, the boundaries of each interval can be defined more accurately to divide the entire data into several statistically significant intervals.

[0081] S204, performing cluster analysis on the plurality of data intervals to obtain a plurality of data groups, wherein the number of data groups is less than or equal to the number of data intervals.

[0082] Specifically, cluster analysis is a further processing of the binned data intervals, aiming to merge intervals with similar characteristics to form several data groups. The goal of cluster analysis is to identify potential groups or patterns in the data, reduce data redundancy, and make the data more interpretable.

[0083] In this step, commonly used clustering algorithms include K-Means, Hierarchical Clustering, and density-based DBSCAN algorithm. K-Means divides data into K preset clusters through iterative optimization, maximizing the similarity within each cluster; Hierarchical clustering merges data intervals level by level by building a clustering tree, which is suitable for situations with uncertain number of groups; and DBSCAN can identify density clusters and outliers in the data, which is very suitable for non-uniform data.

[0084] In specific operations, a data interval can generate a feature vector based on its statistical characteristics (such as mean, variance, etc.) as the input of the clustering algorithm. The result of clustering is to merge multiple data intervals into several more meaningful data groups, where each group represents a class of people or data points with similar data characteristics.

[0085] It is worth noting that the final number of groups should be less than or equal to the number of initial bins, thereby reducing the complexity of the data and improving the effectiveness of the analysis.

[0086] S205, using a preset visualization tool to perform data visualization transformation on several data groups to generate a data portrait distribution.

[0087] Specifically, data visualization presents the results of cluster analysis intuitively, helping users better understand data distribution and group characteristics. Common visualization tools include bar charts, line charts, pie charts, scatter plots, heat maps, etc. Choose the appropriate visualization method according to the different characteristics of the data. For example, if the data is densely distributed, you can use a kernel density curve or a stacked bar chart to show the distribution of each group; if the data shows a clustering effect, you can use a scatter plot and a heat map to intuitively display the clustering results.

[0088] When generating a portrait distribution, we should not only focus on the overall distribution characteristics of the group, but also display the key characteristics within each group, such as the center of the group, data range, distribution form, etc. In addition, visualization can also integrate interactive functions, allowing users to click, zoom in or filter different groups to further explore data characteristics. Through preset visualization tools, data becomes more readable and interpretable, and users can quickly understand the key patterns and characteristic groups of the data, providing a basis for business decisions.

[0089] In the above embodiment, the present application estimates the distribution smoothing of numerical data through the kernel density algorithm to avoid directly processing the complexity of high-cardinality, continuous-valued data. Then, the binning strategy based on density distribution effectively reduces the number of numerical intervals, making complex data more concise. Subsequently, similar intervals are merged into data groups in combination with cluster analysis to further simplify the data dimension. Finally, an intuitive portrait distribution of numerical type labels is generated through visualization tools, allowing users to efficiently view and analyze, solving the problem of constructing a portrait distribution for high-cardinality numerical data.

[0090] Furthermore, after the step of obtaining the full amount of data for constructing the profile distribution, it also includes:

[0091] Perform data anomaly detection on the entire data to determine the abnormal data values ​​in the entire data;

[0092] Perform data cleaning on all data based on abnormal data values;

[0093] The full amount of data that has completed data cleaning is subjected to data standardization processing to obtain standardized full amount of data.

[0094] In this embodiment, anomaly detection is a key step in data processing, the purpose of which is to find data points that deviate from the normal range, usually detected by statistical methods (such as standard deviation, box plot method, Z-score, etc.) or machine learning algorithms (such as isolation forest, LOF). The detected outliers may be caused by input errors, sensor failures or other noise factors, which will interfere with subsequent analysis. Data cleaning includes deleting abnormal data or repairing outliers by methods such as interpolation to ensure the integrity and accuracy of the data. Next, the cleaned data needs to be standardized, usually using methods such as Z-score standardization or Min-Max normalization to convert values ​​of different dimensions or ranges to the same scale. Standardization can improve the performance of data in models such as clustering and regression, and reduce the impact of dimensional differences on the algorithm.

[0095] Through the above steps, the outliers that do not conform to the normal distribution are effectively removed through the data anomaly detection and cleaning steps, the quality of the data is improved, and the model is prevented from being biased due to differences in data range, thereby improving the accuracy and reliability of data analysis.

[0096] Furthermore, the step of using the kernel density estimation algorithm to calculate the density distribution of the entire data specifically includes:

[0097] Select a kernel function that matches the full data set and define the range of the initial bandwidth.

[0098] Within the range of the initial bandwidth, the cross-validation method is used to optimize the bandwidth parameters to obtain the optimal bandwidth parameters;

[0099] The optimal bandwidth parameter is used as the bandwidth value of the kernel density calculation, and the density distribution curve of the full data is drawn using the preset visual density curve drawing tool combined with the kernel function;

[0100] Get the density distribution of the full data from the density distribution curve.

[0101] In this embodiment, we first need to select a kernel function that matches the distribution characteristics of the full data. Common kernel functions include Gaussian kernels, rectangular kernels, triangular kernels, etc., and each kernel function has its applicable scenario. For general continuous data, Gaussian kernels are usually used because they can produce smooth density estimates. When defining the initial bandwidth range, it is necessary to consider the distribution width of the data, and the initial range can be set relatively broadly to ensure that reasonable bandwidth values ​​are covered. Next, the optimal bandwidth is selected within the initial bandwidth range by cross-validation. The cross-validation method divides the data into multiple training sets and validation sets, gradually tests different bandwidth values, and finally determines which bandwidth can obtain the best density estimation result on the validation set.

[0102] After the bandwidth optimization is completed, it is used as the final parameter of the kernel density estimation. Next, a visualization tool is used to draw a density distribution curve. The density distribution curve can intuitively display the probability distribution of the entire data and help users identify the concentrated intervals, peaks, and sparse areas of the data.

[0103] Through the above steps, the density estimation obtains the optimal bandwidth parameter, ensuring the smoothness and accuracy of the distribution curve. The final density distribution curve can clearly show the distribution form of the data and help users deeply understand the concentration trend and discreteness of the data.

[0104] Furthermore, the formula for bandwidth parameter optimization is as follows:

[0105]

[0106] In the formula, σ is the standard deviation of the full data, and n is the number of samples.

[0107] In this embodiment, a cross-validation method is used to automatically select the optimal bandwidth, which is generally a more accurate method. A bandwidth search space can be defined, and a bandwidth that minimizes the density estimation error can be found through cross-validation.

[0108] For example, you can use Silverman's rule of thumb to optimize the bandwidth parameter. Silverman's rule of thumb is a method for selecting the bandwidth parameter in kernel density estimation. Silverman's rule of thumb is a commonly used heuristic method for estimating the optimal bandwidth by calculating a default bandwidth based on the standard deviation of the data and the sample size.

[0109] Furthermore, the steps of binning the full amount of data based on the density distribution to obtain several data intervals specifically include:

[0110] Determine the peak and valley points of the density distribution curve according to the density distribution of the full amount of data;

[0111] Determine the inflection point of density change according to the peak and valley points of the density distribution curve;

[0112] Set the data binning interval according to the inflection point of density change;

[0113] The full amount of data is binned according to the data binning interval to obtain several data intervals.

[0114] In this embodiment, first, a density distribution curve of the full amount of data is generated by a kernel density estimation method, and this curve shows the probability density of the data in different intervals. The peak (i.e., the highest point of density) and the valley (i.e., the lowest point of density) are the key features of the density distribution curve, reflecting the concentrated and sparse areas of the data. By analyzing these peaks and valleys, significant inflection points of density change can be identified, i.e., points where the density changes from high to low or from low to high. These inflection points indicate the natural dividing lines of data distribution, thereby helping to define the data binning intervals. The setting of the binning intervals should take these inflection points into account to ensure that each interval can reasonably reflect the different distribution characteristics of the data. After the binning intervals are determined, the full amount of data is binned, and the data is divided into these intervals to form several data intervals.

[0115] Through the above steps, the data binning process sets the intervals according to the inflection points of the density distribution curve, ensuring the rationality and representativeness of the binning. It can not only effectively reveal the distribution characteristics of the data, but also divide the data into intervals with practical significance.

[0116] Furthermore, the steps of performing cluster analysis on a plurality of data intervals to obtain a plurality of data groups specifically include:

[0117] Perform feature analysis on each data interval to obtain interval data features of each data interval;

[0118] Calculate the data feature similarity between each data interval based on the interval data feature;

[0119] Cluster analysis is performed on several data intervals according to the similarity of data features to obtain several data groups.

[0120] In this embodiment, each data interval is first subjected to feature analysis to extract key data features of each interval, such as mean, variance, data range, density distribution, etc. These features can fully describe the data distribution of each interval. Then, the similarity between intervals is calculated using these features. Commonly used methods include Euclidean distance, Manhattan distance or cosine similarity, etc. These similarity metrics can quantify the degree of similarity between different data intervals. Then, based on the calculated similarity, a cluster analysis algorithm is used to group the data intervals. The goal of cluster analysis is to merge similar intervals into data groups so that the data intervals within the same group have a higher similarity, while the similarity between different groups is lower. Ultimately, the several data groups obtained by clustering can more clearly reflect the inherent structure and relationship of the data.

[0121] Through the above steps, data intervals can be effectively grouped according to feature similarities, revealing the potential structure of the data. This not only improves the accuracy of data analysis, but also provides a clear classification basis for data clustering through cluster analysis.

[0122] Furthermore, after performing cluster analysis on a plurality of data intervals to obtain a plurality of data groups, the method further includes:

[0123] Several data groups are stored in preset application configuration libraries respectively.

[0124] In this embodiment, once a number of data groups are obtained through cluster analysis, the system systematically stores the group information in a preset application configuration library. The application configuration library is usually a database or data warehouse that is specifically used to manage and store application configuration data, analysis results, and other business-related data. In this step, each data group is assigned a unique identifier, and the relevant group information (such as group characteristics, interval range, sample data, etc.) is stored in the corresponding field in the configuration library.

[0125] The storage structure of data groups can be designed in a table format, where each row represents a data group and the columns contain the group's characteristic information and other related attributes. The storage process may include data formatting, compression, and security settings to ensure data integrity and access efficiency. After storage is completed, the application configuration library can support further query, retrieval, and analysis.

[0126] Through the above steps, data groups are systematically stored in the application configuration library, ensuring the persistence and structuring of data and improving the efficiency of data management.

[0127] The steps of using preset visualization tools to perform data visualization transformation on several data groups and generate data portrait distribution include:

[0128] Receive portrait building instructions and determine the data viewing permissions of the initiator of the instructions;

[0129] Read the data group that matches the data viewing permission in the application configuration library to obtain the target data group;

[0130] Use visualization tools to perform data visualization transformation on the target data group and generate data portrait distribution.

[0131] In this embodiment, the system first receives the portrait building instruction initiated by the user and verifies the identity of the instruction initiator and his data viewing authority. This authority verification step ensures that only authorized users can access and operate the data to protect the security and privacy of the data. Next, the system reads the matching data groups from the application configuration library according to the user's authority. These data groups have been screened according to the user's authority settings, with the goal of obtaining the specific data groups that the user has the right to view. Then, these target data groups are visualized using visualization tools. Visualization tools can generate various charts and graphs, such as histograms, density maps, scatter plots, etc., to clearly show the distribution and characteristics of the data groups. The data portrait distribution generation process converts data into an intuitive visual form, allowing users to effectively analyze and understand the data.

[0132] Through the above steps, the system can ensure secure access to data and generate personalized data portrait distribution based on user permissions. This method not only protects data privacy, but also provides easy-to-understand visual reports.

[0133] In the above embodiment, the present application discloses a method for constructing a portrait distribution of a numerical type label, which belongs to the field of big data technology. The present application obtains the full amount of data used to construct the portrait distribution, wherein the full amount of data is numerical type data; uses the kernel density estimation algorithm to calculate the density distribution of the full amount of data; based on the density distribution, the full amount of data is binned to obtain several data intervals; cluster analysis is performed on several data intervals to obtain several data groups, wherein the number of data groups is less than or equal to the number of data intervals; and a preset visualization tool is used to perform data visualization transformation on several data groups to generate a data portrait distribution. The present application estimates the distribution smoothing of numerical data through the kernel density algorithm to avoid directly processing the complexity of data with high cardinality and continuous values, and then the binning strategy based on the density distribution effectively reduces the number of numerical intervals, making complex data more concise, and then combines cluster analysis to merge similar intervals into data groups, further simplifying the data dimension, and finally generates an intuitive portrait distribution of numerical type labels through a visualization tool, so that users can efficiently view and analyze, solving the problem of constructing a portrait distribution for processing high cardinality numerical data.

[0134] In this embodiment, the method for constructing a distribution of a portrait of a numerical type tag is executed on an electronic device (eg Figure 1The server shown in the figure can receive instructions or obtain data through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G connection, Wi-Fi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (Ultra Wi-Fi Band) connection, and other wireless connection methods currently known or to be developed in the future.

[0135] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned full data information, the above-mentioned full data information can also be stored in a node of a blockchain.

[0136] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0137] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0138] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0139] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0140] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0141] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a device for constructing a distribution image of a numerical type label. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0142] like Figure 3 As shown, the image distribution construction device 300 of the numerical type label described in this embodiment includes:

[0143] The data acquisition module 301 is used to acquire the full amount of data used to construct the profile distribution, wherein the full amount of data is numerical type data;

[0144] Density calculation module 302, used to calculate the density distribution of the entire data using a kernel density estimation algorithm;

[0145] The data binning module 303 is used to bin the full amount of data based on the density distribution to obtain a number of data intervals;

[0146] A cluster analysis module 304 is used to perform cluster analysis on a plurality of data intervals to obtain a plurality of data groups, wherein the number of data groups is less than or equal to the number of data intervals;

[0147] The portrait distribution module 305 is used to use a preset visualization tool to perform data visualization transformation on a plurality of data groups to generate a data portrait distribution.

[0148] Furthermore, the image distribution construction device 300 of the numerical type label also includes a data preprocessing module, which is specifically used for:

[0149] Perform data anomaly detection on the full amount of data to determine the abnormal data values ​​in the full amount of data; perform data cleaning on the full amount of data based on the abnormal data values; perform data standardization processing on the full amount of data that has completed data cleaning to obtain standardized full amount of data.

[0150] Further, the density calculation module 302 is specifically used for:

[0151] Select a kernel function that matches the full amount of data and define the range of the initial bandwidth; within the range of the initial bandwidth, use the cross-validation method to optimize the bandwidth parameters to obtain the optimal bandwidth parameters; use the optimal bandwidth parameters as the bandwidth value for kernel density calculation, and use the preset visual density curve drawing tool combined with the kernel function to draw the density distribution curve of the full amount of data; obtain the density distribution of the full amount of data from the density distribution curve.

[0152] Furthermore, the formula for bandwidth parameter optimization is as follows:

[0153]

[0154] In the formula, σ is the standard deviation of the full data, and n is the number of samples.

[0155] Furthermore, the data binning module 303 is specifically used for:

[0156] Determine the peak and valley points of the density distribution curve according to the density distribution of the full data; determine the density change inflection point according to the peak and valley points of the density distribution curve; set the data binning interval according to the density change inflection point; bin the full data according to the data binning interval to obtain several data intervals.

[0157] Furthermore, the cluster analysis module 304 is specifically used for:

[0158] Perform feature analysis on each data interval to obtain interval data features of each data interval; calculate the data feature similarity between each data interval based on the interval data features; perform cluster analysis on several data intervals according to the data feature similarity to obtain several data groups.

[0159] Furthermore, the image distribution construction device 300 of the numerical type label also includes a data storage module, which is specifically used for:

[0160] Several data groups are stored in preset application configuration libraries respectively.

[0161] Furthermore, the portrait distribution module 305 is specifically used for:

[0162] Receive the portrait building instruction and determine the data viewing permission of the instruction initiator; read the data group that matches the data viewing permission in the application configuration library to obtain the target data group; use the visualization tool to perform data visualization conversion on the target data group to generate a data portrait distribution.

[0163] In the above embodiment, the present application discloses a device for constructing a portrait distribution of a numerical type label, which belongs to the field of big data technology. The present application obtains the full amount of data used to construct the portrait distribution, wherein the full amount of data is numerical type data; uses the kernel density estimation algorithm to calculate the density distribution of the full amount of data; based on the density distribution, the full amount of data is binned to obtain several data intervals; cluster analysis is performed on several data intervals to obtain several data groups, wherein the number of data groups is less than or equal to the number of data intervals; and a preset visualization tool is used to perform data visualization transformation on several data groups to generate a data portrait distribution. The present application estimates the distribution smoothing of numerical data through the kernel density algorithm to avoid directly processing the complexity of data with high cardinality and continuous values, and then the binning strategy based on the density distribution effectively reduces the number of numerical intervals, making complex data more concise, and then combines cluster analysis to merge similar intervals into data groups, further simplifying the data dimension, and finally generates an intuitive portrait distribution of numerical type labels through a visualization tool, so that users can efficiently view and analyze, solving the problem of constructing a portrait distribution for processing high cardinality numerical data.

[0164] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0165] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 4 having a memory 41, a processor 42, and a network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0166] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0167] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Med ia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions of the image distribution construction method of the numerical type label, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0168] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as running the computer-readable instructions of the method for constructing the image distribution of the numerical type label.

[0169] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0170] In the above embodiment, the present application discloses a computer device, which belongs to the field of big data technology. The present application obtains the full amount of data used to construct a portrait distribution, wherein the full amount of data is numerical type data; uses a kernel density estimation algorithm to calculate the density distribution of the full amount of data; based on the density distribution, the full amount of data is binned to obtain several data intervals; cluster analysis is performed on several data intervals to obtain several data groups, wherein the number of data groups is less than or equal to the number of data intervals; and a preset visualization tool is used to perform data visualization transformation on several data groups to generate a data portrait distribution. The present application estimates the distribution smoothing of numerical data through a kernel density algorithm to avoid directly processing the complexity of data with high cardinality and continuous values, and then the binning strategy based on density distribution effectively reduces the number of numerical intervals, making complex data more concise, and then combines cluster analysis to merge similar intervals into data groups, further simplifying the data dimension, and finally generates an intuitive portrait distribution of numerical type labels through a visualization tool, so that users can efficiently view and analyze, solving the problem of constructing a portrait distribution for processing high cardinality numerical data.

[0171] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the method for constructing a portrait distribution of numerical type labels as described above.

[0172] In the above embodiment, the present application discloses a computer-readable storage medium, which belongs to the field of big data technology. The present application obtains the full amount of data used to construct the portrait distribution, wherein the full amount of data is numerical type data; uses the kernel density estimation algorithm to calculate the density distribution of the full amount of data; based on the density distribution, the full amount of data is binned to obtain several data intervals; cluster analysis is performed on several data intervals to obtain several data groups, wherein the number of data groups is less than or equal to the number of data intervals; and a preset visualization tool is used to perform data visualization transformation on several data groups to generate a data portrait distribution. The present application estimates the distribution smoothing of numerical data through the kernel density algorithm to avoid directly processing the complexity of high cardinality and continuous value data, and then the binning strategy based on the density distribution effectively reduces the number of numerical intervals, making complex data more concise, and then combines cluster analysis to merge similar intervals into data groups, further simplifying the data dimension, and finally generates an intuitive portrait distribution of numerical type labels through a visualization tool, so that users can efficiently view and analyze, solving the problem of constructing a portrait distribution for high cardinality numerical data.

[0173] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0174] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0175] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for constructing a distribution image of a numerical type label, characterized in that: include: Acquire the full amount of data used to construct the profile distribution, wherein the full amount of data is numerical type data; Calculate the density distribution of the full amount of data using a kernel density estimation algorithm; Based on the density distribution, the full amount of data is binned to obtain a plurality of data intervals; Performing cluster analysis on a number of the data intervals to obtain a number of data groups, wherein the number of the data groups is less than or equal to the number of the data intervals; Use preset visualization tools to perform data visualization transformation on several of the data groups to generate data portrait distribution.

2. The method for constructing a distribution image of a numerical type label according to claim 1, characterized in that: After the step of obtaining the full amount of data for constructing the portrait distribution, the method further includes: Performing data anomaly detection on the full amount of data to determine abnormal data values ​​in the full amount of data; Performing data cleaning on the full amount of data based on the abnormal data value; The full amount of data that has completed data cleaning is subjected to data standardization processing to obtain standardized full amount of data.

3. The method for constructing a distribution image of a numerical type label according to claim 1, characterized in that: The step of using a kernel density estimation algorithm to calculate the density distribution of the full amount of data specifically includes: Selecting a kernel function that matches the full amount of data and defining a range of an initial bandwidth; Within the range of the initial bandwidth, a cross-validation method is used to optimize the bandwidth parameters to obtain the optimal bandwidth parameters; The optimal bandwidth parameter is used as the bandwidth value of the kernel density calculation, and a preset visual density curve drawing tool is used in combination with the kernel function to draw a density distribution curve of the full amount of data; The density distribution of the total amount of data is obtained from the density distribution curve.

4. The method for constructing a distribution image of a numerical type label according to claim 3, characterized in that: The formula for bandwidth parameter optimization is as follows: In the formula, σ is the standard deviation of the full data, and n is the number of samples.

5. The method for constructing a distribution image of a numerical type label according to claim 3, characterized in that: The step of binning the full amount of data based on the density distribution to obtain a plurality of data intervals specifically includes: Determining the peak and valley points of the density distribution curve according to the density distribution of the full amount of data; Determining the density change inflection point according to the peak and valley points of the density distribution curve; Setting a data binning interval according to the density change inflection point; The full amount of data is binned according to the data binning intervals to obtain a plurality of the data intervals.

6. The method for constructing a distribution image of a numerical type label according to claim 1, characterized in that: The step of performing cluster analysis on the plurality of data intervals to obtain a plurality of data groups specifically includes: Performing feature analysis on each of the data intervals to obtain interval data features of each of the data intervals; Calculating the data feature similarity between each of the data intervals based on the interval data features; Cluster analysis is performed on a plurality of the data intervals according to the similarity of the data features to obtain a plurality of the data groups.

7. The method for constructing a distribution image of a numerical type label according to claim 1, characterized in that: After the step of performing cluster analysis on the plurality of data intervals to obtain a plurality of data groups, the method further includes: storing the plurality of data groups into a preset application configuration library respectively; The step of using a preset visualization tool to perform data visualization transformation on the plurality of data groups to generate data portrait distribution specifically includes: Receive portrait building instructions and determine the data viewing permissions of the initiator of the instructions; Reading a data group matching the data viewing permission in the application configuration library to obtain a target data group; The visualization tool is used to perform data visualization transformation on the target data group to generate the data portrait distribution.

8. A device for constructing a distribution image of a numerical type label, characterized in that: include: A data acquisition module, used to acquire the full amount of data used to construct the portrait distribution, wherein the full amount of data is numerical type data; A density calculation module, used to calculate the density distribution of the full amount of data using a kernel density estimation algorithm; A data binning module, used to bin the full amount of data based on the density distribution to obtain a plurality of data intervals; A cluster analysis module, used for performing cluster analysis on the plurality of data intervals to obtain a plurality of data groups, wherein the number of the data groups is less than or equal to the number of the data intervals; The image distribution module is used to use a preset visualization tool to perform data visualization transformation on a number of the data groups to generate a data image distribution.

9. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, it implements the steps of the method for constructing a portrait distribution of a numerical type label as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the method for constructing a distribution image of a numerical type label as described in any one of claims 1 to 7.

Images