Digital intelligent environmental sanitation data optimization storage method and device and storage medium

Abstract

The invention is suitable for the technical field of data storage, and provides a digital intelligent environmental sanitation data optimization storage method and device and a storage medium, the digital intelligent environmental sanitation data optimization storage method comprises the following steps: calculating feature data corresponding to different sample environmental sanitation data in order to carry out classified storage on different environmental sanitation data, and on the basis of the feature data, analyzing a conversion relationship between different feature data, and further calculating a clustering center of the plurality of clustering clusters according to the probability distribution diagram. And taking different clustering centers as an index structure, and storing the to-be-stored environmental sanitation data corresponding to each clustering center to a respective corresponding storage area under the index structure. According to the scheme, through clustering of the feature data, automatic classification of the environmental sanitation data is achieved, similar data are stored in the same storage area, storage of the environmental sanitation data is optimized, storage reasonability is improved, and query efficiency is also improved in the subsequent query process.

Description

Technical Field

[0001] The present invention belongs to the technical field of data storage, and in particular, relates to a method and device for optimizing storage of digital intelligent sanitation data. Background Art

[0002] Digital smart sanitation data refers to the digital processing and intelligent application of various data involved in sanitation work to improve sanitation efficiency, reduce resource waste, and support decision-making and management. The main contents of digital sanitation data include: Geographic Information System (GIS) data: Mark the relevant location information of sanitation facilities, routes, garbage bins, sewage outlets, etc. on the map to provide spatial reference and resource distribution. Garbage classification data: Record the classification of garbage by residents or enterprises according to different categories, so as to carry out garbage classification and resource utilization. Garbage collection data: Record the driving route of garbage vehicles, the amount of garbage collected each time, the collection time and other information, so as to dispatch and optimize the use of sanitation vehicles. Cleaning operation data: Record the operating scope, operating time, cleaning volume and other information of cleaning personnel to help understand the completion of cleaning tasks and human resource allocation. Environmental monitoring data: Including monitoring data of environmental parameters such as noise, PM2.5, and air quality, which are used to evaluate the impact of sanitation work on the environment and take corresponding improvement measures. Equipment operation data: records the working status and fault information of sanitation equipment (such as trash compactors, sweeping robots, etc.) for equipment maintenance and management.

[0003] Since smart sanitation data is huge and complex, the storage of smart sanitation data poses a challenge. How to optimize the storage of smart sanitation data has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a digital smart sanitation data optimization storage method, device and storage medium to solve the technical problem of how to optimize the storage of smart sanitation data.

[0005] A first aspect of an embodiment of the present invention provides a method for optimizing storage of digital smart sanitation data, the method comprising:

[0006] The first calculation step: calculating feature data corresponding to the sample sanitation data, and calculating a first similarity between each of the feature data; the sample sanitation data is used to train a plurality of cluster centers;

[0007] The second calculation step: calculating the conversion probability distribution between each feature data according to the first similarity; the conversion probability distribution refers to the probability of mutual conversion between two feature data;

[0008] The first construction step: constructing a probability matrix according to the conversion probability distribution; wherein each column element is a conversion probability distribution between a single feature data and a plurality of other feature data;

[0009] The third calculation step: performing eigenvalue decomposition on the probability matrix to obtain a probability matrix after dimensionality reduction;

[0010] The second construction step: constructing a probability distribution graph based on the probability matrix;

[0011] Extraction step: extracting clusters in the probability distribution graph and calculating cluster centers of the clusters; all nodes in the clusters are connected to one or more nodes in the clusters;

[0012] Storage step: using the cluster center of the cluster cluster as an index structure, and storing the to-be-stored sanitation data corresponding to each cluster center into the storage area corresponding to each of the index structures.

[0013] Furthermore, the storing step includes:

[0014] The first calculation sub-step: calculating the second similarity between the multiple cluster centers;

[0015] Determining sub-step: if the second similarity is greater than a preset similarity, the cluster centers corresponding to the second similarity are fused to obtain a fused cluster center;

[0016] The first selection sub-step: using the fused cluster center and other cluster centers as the index structure respectively; the other cluster centers refer to cluster centers that have not been fused;

[0017] Storage sub-step: storing the to-be-stored sanitation data corresponding to each cluster center into the respective corresponding storage areas under the index structure.

[0018] Furthermore, after the storing step, the method further comprises:

[0019] Second calculation sub-step: obtaining a query statement or a query condition, and calculating a feature vector corresponding to the query statement or the query condition;

[0020] The third calculation sub-step: calculating the third similarity between the feature vector and the fusion cluster center and other cluster centers in the index structure;

[0021] Second selection sub-step: taking the storage area corresponding to the index structure of the third similarity as the target storage area;

[0022] Retrieval sub-step: in the target storage area, retrieve the target sanitation data corresponding to the query statement or query condition.

[0023] Furthermore, the second calculating step includes:

[0024] Substituting the first similarity into the following calculation formula, the conversion probability distribution between each feature data is obtained;

[0025]

[0026] Among them, P(ij) represents the conversion probability distribution between the i-th feature data and the j-th feature data, x i and x j Represents two feature data, ||x i -x j || represents the first similarity between two feature data, ∑k≠i represents the similarity between two feature data x i Sum all feature data except x k represents the kth feature data among all feature data, represents the average value of all characteristic data, and σ represents the adjustment coefficient.

[0027] Furthermore, the third calculation step includes:

[0028] Fourth calculation sub-step: calculating the covariance matrix corresponding to the probability matrix;

[0029] The fifth calculation sub-step: performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvectors and a plurality of eigenvalues;

[0030] Arrangement sub-step: arranging the plurality of eigenvalues ​​and selecting eigenvectors corresponding to the first k eigenvalues;

[0031] The first construction sub-step: constructing the feature vectors into a projection matrix in order;

[0032] The sixth calculation sub-step: multiplying the probability matrix by the projection matrix to obtain a probability matrix after dimensionality reduction.

[0033] Furthermore, the extraction step comprises:

[0034] Selection sub-step: select any node in the distribution graph and obtain the current node connected to the node;

[0035] The first acquisition sub-step: acquiring subsequent nodes connected to the current node;

[0036] Repeat the sub-steps: taking the subsequent node as the current node, and repeatedly performing the step of obtaining the subsequent node connected to the current node until there is no new node connected, thereby obtaining the cluster;

[0037] The seventh calculation sub-step: calculating the mean of the characteristic data in the cluster to obtain the cluster center.

[0038] Furthermore, the second construction step includes:

[0039] The second acquisition sub-step: obtaining the value of each element in the probability matrix after dimensionality reduction and the feature data corresponding to each element, and constructing an initial distribution graph based on the value of each element and the two feature data corresponding to each element; the nodes of the distribution graph are the feature data, and the edges between the nodes are the values ​​corresponding to the two feature data;

[0040] Screening sub-step: removing edges whose values ​​in the initial distribution graph are lower than a threshold, and removing isolated nodes in the initial distribution graph, to obtain the probability distribution graph.

[0041] A second aspect of an embodiment of the present invention provides a digital intelligent sanitation data optimization storage device, comprising:

[0042] A first calculation unit is used to calculate feature data corresponding to the sample sanitation data and calculate a first similarity between each of the feature data; the sample sanitation data is used to train a plurality of cluster centers;

[0043] A second calculation unit is used to calculate the conversion probability distribution between each feature data according to the first similarity; the conversion probability distribution refers to the probability of mutual conversion between two feature data;

[0044] A first construction unit is used to construct a probability matrix according to the conversion probability distribution, wherein each column element is a conversion probability distribution between a single feature data and a plurality of other feature data;

[0045] A third computing unit is used to perform eigenvalue decomposition on the probability matrix to obtain a probability matrix after dimensionality reduction;

[0046] A second construction unit is used to construct a probability distribution graph based on the probability matrix;

[0047] An extraction unit, used to extract clusters in the probability distribution graph and calculate cluster centers of the clusters; all nodes in the clusters are connected to one or more nodes in the clusters;

[0048] The storage unit is used to use the cluster center of the cluster cluster as an index structure, and store the sanitation data to be stored corresponding to each cluster center into the storage area corresponding to each other under the index structure.

[0049] A third aspect of an embodiment of the present invention provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0050] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0051] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: the present invention calculates the first similarity between each feature data by calculating the feature data corresponding to the sample sanitation data; the sample sanitation data is used to train multiple clustering centers; based on the first similarity, the conversion probability distribution between each feature data is calculated; the conversion probability distribution refers to the probability of mutual conversion between two feature data; based on the conversion probability distribution, a probability matrix is ​​constructed; wherein each column element is the conversion probability distribution between a single feature data and multiple other feature data; the probability matrix is ​​subjected to eigenvalue decomposition to obtain a probability matrix after dimensionality reduction; based on the probability matrix, a probability distribution graph is constructed; clusters in the probability distribution graph are extracted, and the clustering centers of the clusters are calculated; all nodes in the clusters are connected to one or more nodes in the clusters; the clustering centers of the clusters are used as index structures, and the sanitation data to be stored corresponding to each clustering center is stored in the storage areas corresponding to the respective index structures. In the above scheme, in order to classify and store different sanitation data, the characteristic data corresponding to different sample sanitation data are calculated, and the conversion relationship between different characteristic data is analyzed based on the characteristic data, and then the cluster centers of multiple cluster clusters are calculated according to the probability distribution diagram. Different cluster centers are used as index structures, and the sanitation data to be stored corresponding to each cluster center is stored in the storage area corresponding to the index structure. The above scheme realizes the automatic classification of sanitation data through the clustering of characteristic data, and stores similar data in the same storage area, which not only optimizes the storage of sanitation data and improves the rationality of storage, but also improves the query efficiency in the subsequent query process. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1A schematic flow chart of a digital intelligent sanitation data optimization storage method provided by the present invention is shown;

[0054] Figure 2 A schematic diagram of a digital intelligent sanitation data optimization storage device provided by an embodiment of the present invention is shown;

[0055] Figure 3 A schematic diagram of a terminal device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0056] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0057] The embodiments of the present invention provide a method, device and storage medium for optimizing the storage of digital smart sanitation data to solve the technical problem of how to optimize the storage of smart sanitation data.

[0058] First, the present invention provides a method for optimizing the storage of digital smart sanitation data. Figure 1 , Figure 1 FIG. 1 is a schematic flow chart of a digital intelligent sanitation data optimization storage method provided by the present invention. Figure 1 As shown, the digital intelligent sanitation data optimization storage method may include the following steps:

[0059] The first calculation step: calculating feature data corresponding to the sample sanitation data, and calculating a first similarity between each of the feature data; the sample sanitation data is used to train a plurality of cluster centers;

[0060] Since the amount of sanitation data is huge and the classification is complicated, manual classification and storage is highly subjective and is not conducive to improving the efficiency of subsequent data queries. Therefore, this application allocates different storage spaces for different sanitation data through automated classification, and the clustering centers obtained by clustering can also be applied to subsequent data queries, thereby achieving a high degree of coordination between storage and query. This not only optimizes the rationality of data storage, but also improves the efficiency of subsequent data queries.

[0061] The sample sanitation data is artificially preset training data, which is used to train cluster centers for classification. The sample sanitation data includes all types or some types of sanitation data.

[0062] For character-type sample sanitation data, key character retrieval is used to extract key characters, and the key characters are encoded to obtain feature data (vector) composed of multiple encoding values.

[0063] As an embodiment of the present application, since there are large differences between character-type sample sanitation data and image-type sample sanitation data, if there is image-type sample sanitation data, it is necessary to classify and store the character-type sample sanitation data and the image-type sample sanitation data separately for optimization. Among them, for the image-type sample sanitation data, it is necessary to extract the feature data of the image through image processing.

[0064] Each character-type sanitation data is a data group composed of multiple single-point data. For example, the driving route data of garbage trucks in a single sanitation operation includes multiple track point data, vehicle information, operator and other single-point data. The garbage volume data collected in a single sanitation operation includes multiple single-point data such as collection location point, collection garbage bin code and collection quantity. The collection time includes multiple collection time points in a day and multiple single-point data such as number information and location information corresponding to the collection time points.

[0065] The first similarity between each feature data is calculated by Euclidean distance.

[0066] The second calculation step: calculating the conversion probability distribution between each feature data according to the first similarity; the conversion probability distribution refers to the probability of mutual conversion between two feature data;

[0067] The conversion probability distribution is used to extract the similarity features between feature data, and then improve the data basis for subsequent calculations. The larger the value of the conversion probability distribution, the higher the conversion probability between feature data.

[0068] Specifically, the second calculation step includes the following steps:

[0069] Substituting the first similarity into the following calculation formula, the conversion probability distribution between each feature data is obtained;

[0070]

[0071] Among them, P(ij) represents the conversion probability distribution between the i-th feature data and the j-th feature data, x i and x j Represents two feature data, ||x i -x j || represents the first similarity between two feature data, ∑k≠i represents the similarity between two feature data x i Sum all feature data except x k represents the kth feature data among all feature data, Represents the average value of all feature data, and σ represents the adjustment coefficient. A smaller σ value will result in a sharp transition probability distribution, while a larger σ value will produce a smooth transition probability distribution.

[0072] First construction step: construct a probability matrix according to the transition probability distribution; wherein each column element is the transition probability distribution between a single feature data and multiple other feature data; (construct an N×N similarity matrix P, where N is the number of data points in the data set. The element Pij of the matrix represents the transition probability from the data point Xi to Xj)

[0073] For example, it is assumed that the feature data is A1, A2, B1, and B2. The conversion probability distribution calculated based on the feature data is A1A2, A1B1, A1B2, A2A1, A2B1, A2B2, B1A1, B1A2, B1B2, B2A1, B2A2, B2B1. A1A2 represents the conversion probability of A1 to A2, and so on.

[0074] Based on the above transition probabilities, the probability matrix obtained is:

[0075] The third calculation step: performing eigenvalue decomposition on the probability matrix to obtain a probability matrix after dimensionality reduction;

[0076] Specifically, the third calculation step specifically includes a fourth calculation sub-step, a fifth calculation sub-step, an arrangement sub-step, a first construction sub-step and a sixth calculation sub-step:

[0077] Fourth calculation sub-step: calculating the covariance matrix corresponding to the probability matrix;

[0078] The fifth calculation sub-step: performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvectors and a plurality of eigenvalues;

[0079] Arrangement sub-step: arranging the plurality of eigenvalues ​​and selecting eigenvectors corresponding to the first k eigenvalues;

[0080] The first construction sub-step: constructing the feature vectors into a projection matrix in order;

[0081] The sixth calculation sub-step: multiplying the probability matrix by the projection matrix to obtain a probability matrix after dimensionality reduction.

[0082] For example, suppose the probability matrix is ​​X, which contains n samples and m features. The probability matrix after dimensionality reduction is Y, which contains n samples and k principal components (features).

[0083] Each sample is a vector in the probability matrix, usually represented as Xi, where i represents the index of the sample (1 <= i <= n). Similarly, in the probability matrix after dimensionality reduction, each sample is also a vector, represented as Yi.

[0084] The i-th row of the probability matrix represents the feature vector of the i-th sample, denoted as Xi = [x1i, x2i, ..., xmi], where j represents the index of the feature (1 <= j <= m), and xji represents the value of the i-th sample on the j-th feature.

[0085] The i-th row of the probability matrix after dimensionality reduction also represents the feature vector of the i-th sample, denoted as Yi=[y1i,y2i,...,yki], where j represents the index of the feature after dimensionality reduction (1<=j<=k), and yji represents the value of the i-th sample on the j-th feature after dimensionality reduction.

[0086] It should be noted that in the process of dimensionality reduction, the probability matrix is ​​mapped to the reduced dimensionality space through the projection matrix. Therefore, the feature of each sample in the probability matrix after dimensionality reduction is the projection of the probability matrix in the reduced dimensionality space.

[0087] In this embodiment, according to a given probability matrix, the corresponding covariance matrix is ​​calculated. The obtained covariance matrix is ​​subjected to eigenvalue decomposition to obtain multiple eigenvectors and multiple eigenvalues. All eigenvalues ​​are arranged by size, and the eigenvectors corresponding to the first k eigenvalues ​​are selected. The selected eigenvectors are constructed into a projection matrix in order. The original probability matrix is ​​multiplied by the constructed projection matrix to obtain a probability matrix after dimensionality reduction. Through these steps, the original probability matrix can be subjected to dimensionality reduction processing to obtain a new probability matrix after eigenvalue decomposition and projection transformation. The matrix after dimensionality reduction has a lower dimension than the original matrix, but still retains important information. It is of great value for data analysis.

[0088] The second construction step: constructing a probability distribution graph based on the probability matrix; the probability distribution graph is composed of nodes and edges between nodes, the nodes are composed of the feature data, and the edges are composed of the conversion probability distribution;

[0089] The feature data is used as nodes, and the corresponding conversion probability distribution values ​​between nodes are used as edges to obtain a probability distribution graph.

[0090] Specifically, the second construction step specifically includes a second acquisition sub-step and a screening sub-step:

[0091] The second acquisition sub-step: obtaining the value of each element in the probability matrix after dimensionality reduction and the feature data corresponding to each element, and constructing an initial distribution graph based on the value of each element and the two feature data corresponding to each element; the nodes of the distribution graph are the feature data, and the edges between the nodes are the values ​​corresponding to the two feature data;

[0092] Screening sub-step: removing edges whose values ​​in the initial distribution graph are lower than a threshold, and removing isolated nodes in the initial distribution graph, to obtain the probability distribution graph.

[0093] In this embodiment, in order to avoid unnecessary calculation, the edges with values ​​lower than the threshold are removed and the isolated nodes are removed.

[0094] Extraction step: extracting clusters in the probability distribution graph and calculating cluster centers of the clusters; all nodes in the clusters are connected to one or more nodes in the clusters;

[0095] Traverse all nodes in the probability distribution graph, extract nodes with connection relationships, and obtain multiple clusters.

[0096] Specifically, the extraction step specifically includes a selection sub-step, a first acquisition sub-step, a repeated execution sub-step and a seventh calculation sub-step:

[0097] Selection sub-step: select any node in the distribution graph and obtain the current node connected to the node;

[0098] The first acquisition sub-step: acquiring subsequent nodes connected to the current node;

[0099] Repeat the sub-steps: taking the subsequent node as the current node, and repeatedly performing the step of obtaining the subsequent node connected to the current node until there is no new node connected, thereby obtaining the cluster;

[0100] The seventh calculation sub-step: calculating the mean of the characteristic data in the cluster to obtain the cluster center.

[0101] Storage step: using the cluster center of the cluster cluster as an index structure, and storing the to-be-stored sanitation data corresponding to each cluster center into the storage area corresponding to each of the index structures.

[0102] For the sanitation data to be stored, feature data of the sanitation data to be stored is extracted, and the distance between the feature data and multiple cluster centers is calculated. The target cluster center corresponding to the minimum distance is selected, and the sanitation data to be stored is stored in the storage area under the index structure corresponding to the target cluster center.

[0103] Specifically, the storage step includes a first calculation sub-step, a judgment sub-step, a first selection sub-step and a storage sub-step:

[0104] The first calculation sub-step: calculating the second similarity between the multiple cluster centers;

[0105] Determining sub-step: if the second similarity is greater than a preset similarity, the cluster centers corresponding to the second similarity are fused to obtain a fused cluster center;

[0106] The first selection sub-step: using the fused cluster center and other cluster centers as the index structure respectively; the other cluster centers refer to cluster centers that have not been fused;

[0107] Storage sub-step: storing the to-be-stored sanitation data corresponding to each cluster center into the respective corresponding storage areas under the index structure.

[0108] After multiple cluster centers are obtained based on sample sanitation data training, the sanitation data to be stored needs to be stored in the storage areas corresponding to each other under the index structure based on the multiple cluster centers, where different cluster centers correspond to different storage areas.

[0109] In this embodiment, in order to improve the rationality of the classification of cluster centers, similar cluster centers are merged.

[0110] Optionally, after the storing step, the method further includes a second calculating sub-step, a third calculating sub-step, a second selecting sub-step and a retrieving sub-step:

[0111] Second calculation sub-step: obtaining a query statement or a query condition, and calculating a feature vector corresponding to the query statement or the query condition;

[0112] The third calculation sub-step: calculating the third similarity between the feature vector and the fusion cluster center and other cluster centers in the index structure;

[0113] Second selection sub-step: taking the storage area corresponding to the index structure of the third similarity as the target storage area;

[0114] The fused cluster center or other cluster centers corresponding to the third similarity are obtained, and the storage area of ​​the index structure corresponding to the fused cluster center or other cluster centers is used as the target storage area.

[0115] Retrieval sub-step: in the target storage area, retrieve the target sanitation data corresponding to the query statement or query condition.

[0116] The above scheme can quickly generate the relevant index by calculating the feature vector corresponding to the query statement or query condition. The third similarity between the feature vector and the fusion cluster center and other cluster centers is calculated to improve the accuracy of the result. The target storage area is determined according to the third similarity to avoid invalid search and reduce the retrieval time. By optimizing the index structure, the speed of data retrieval can be accelerated and the performance of the overall system can be improved.

[0117] In this embodiment, by calculating the feature data corresponding to the sample sanitation data, the first similarity between each of the feature data is calculated; the sample sanitation data is used to train multiple clustering centers; based on the first similarity, the conversion probability distribution between each feature data is calculated; the conversion probability distribution refers to the probability of mutual conversion between two feature data; based on the conversion probability distribution, a probability matrix is ​​constructed; wherein each column element is the conversion probability distribution between a single feature data and multiple other feature data; the probability matrix is ​​subjected to eigenvalue decomposition to obtain a probability matrix after dimensionality reduction; based on the probability matrix, a probability distribution graph is constructed; the clusters in the probability distribution graph are extracted, and the clustering centers of the clusters are calculated; all nodes in the clusters are connected to one or more nodes in the clusters; the clustering centers of the clusters are used as index structures, and the sanitation data to be stored corresponding to each cluster center is stored in the storage areas corresponding to the respective index structures. In the above scheme, in order to classify and store different sanitation data, the characteristic data corresponding to different sample sanitation data are calculated, and the conversion relationship between different characteristic data is analyzed based on the characteristic data, and then the cluster centers of multiple cluster clusters are calculated according to the probability distribution diagram. Different cluster centers are used as index structures, and the sanitation data to be stored corresponding to each cluster center is stored in the storage area corresponding to the index structure. The above scheme realizes the automatic classification of sanitation data through the clustering of characteristic data, and stores similar data in the same storage area, which not only optimizes the storage of sanitation data and improves the rationality of storage, but also improves the query efficiency in the subsequent query process.

[0118] like Figure 2 The present invention provides a digital intelligent sanitation data optimization storage device 2, see Figure 2 , Figure 2 A schematic diagram of a digital intelligent sanitation data optimization storage device provided by the present invention is shown. Figure 2 A digital intelligent sanitation data optimization storage device is shown, comprising:

[0119] A first calculation unit 21 is used to calculate feature data corresponding to sample sanitation data and calculate a first similarity between each of the feature data; the sample sanitation data is used to train multiple cluster centers;

[0120] A second calculation unit 22 is used to calculate the conversion probability distribution between each feature data according to the first similarity; the conversion probability distribution refers to the probability of mutual conversion between two feature data;

[0121] A first construction unit 23 is used to construct a probability matrix according to the conversion probability distribution, wherein each column element is a conversion probability distribution between a single feature data and a plurality of other feature data;

[0122] A third calculation unit 24 is used to perform eigenvalue decomposition on the probability matrix to obtain a probability matrix after dimensionality reduction;

[0123] A second construction unit 25 is used to construct a probability distribution graph based on the probability matrix; the probability distribution graph is composed of nodes and edges between nodes, the nodes are composed of the feature data, and the edges are composed of the conversion probability distribution;

[0124] An extraction unit 26, used to extract clusters in the probability distribution graph and calculate the cluster centers of the clusters; all nodes in the clusters are connected to one or more nodes in the clusters;

[0125] The storage unit 27 is used to use the cluster center of the cluster cluster as an index structure, and store the to-be-stored sanitation data corresponding to each cluster center into the storage area corresponding to each of the index structures.

[0126] The present invention provides a digital intelligent sanitation data optimization storage device, which sends a data encryption request and destination terminal information to an encryption server through a sending end; receives the encryption information returned by the encryption server according to the destination terminal information; the encryption information includes a random value and an encryption identifier; obtains the MAC address of the destination terminal, and converts the MAC address of the destination terminal into a hexadecimal positive integer; according to the encryption identifier, the random value and the positive integer are input into an encryption function to obtain a key sequence output by the encryption function; according to the number of keys in the key sequence, the plaintext data is divided into multiple data blocks; the number of the multiple data blocks is the same as the number of keys in the key sequence; according to the key sequence, the multiple data blocks are encrypted to obtain multiple ciphertext data; the multiple ciphertext data are combined in sequence to obtain the target data, and the target data is sent to the destination terminal. In the above scheme, the sending end and the receiving end respectively request the same random value from the encryption server to prevent data theft in the transmission between the sending end and the receiving end. Then, the key sequence is calculated based on the encryption identifier and the random value, the data is divided into multiple blocks and encrypted using different keys, which increases the security of the data. Finally, the encrypted data is combined in order and sent to the destination terminal, thus ensuring the confidentiality and integrity of the financial data during transmission. Since the random value acquisition process is decoupled and the key sequence is calculated based on the encryption identifier and the random value, the difficulty of deciphering the encrypted data is greatly increased, thereby improving the security of financial data transmission.

[0127] Figure 3 Schematic diagram of a terminal device provided by an embodiment of the present invention. Figure 3 As shown, a terminal device 3 of this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for optimizing storage of digital smart sanitation data. When the processor 30 executes the computer program 32, the steps in each of the above-mentioned embodiments of the method for optimizing storage of digital smart sanitation data are implemented, such as Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of each unit in the above-mentioned device embodiments are realized, for example Figure 2 The functions of units 21 to 27 are shown.

[0128] Exemplarily, the computer program 32 may be divided into one or more units, which are stored in the memory 31 and executed by the processor 30 to implement the present invention. The one or more units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 32 in the terminal device 3.

[0129] For example, the computer program 32 may be divided into units with specific functions as follows:

[0130] A first calculation unit is used to calculate feature data corresponding to the sample sanitation data and calculate a first similarity between each of the feature data; the sample sanitation data is used to train a plurality of cluster centers;

[0131] A second calculation unit is used to calculate the conversion probability distribution between each feature data according to the first similarity; the conversion probability distribution refers to the probability of mutual conversion between two feature data;

[0132] A first construction unit is used to construct a probability matrix according to the conversion probability distribution, wherein each column element is a conversion probability distribution between a single feature data and a plurality of other feature data;

[0133] A third computing unit is used to perform eigenvalue decomposition on the probability matrix to obtain a probability matrix after dimensionality reduction;

[0134] A second construction unit is used to construct a probability distribution graph based on the probability matrix; the probability distribution graph is composed of nodes and edges between nodes, the nodes are composed of the feature data, and the edges are composed of the conversion probability distribution;

[0135] An extraction unit, used to extract clusters in the probability distribution graph and calculate cluster centers of the clusters; all nodes in the clusters are connected to one or more nodes in the clusters;

[0136] The storage unit is used to use the cluster center of the cluster cluster as an index structure, and store the sanitation data to be stored corresponding to each cluster center into the storage area corresponding to each other under the index structure.

[0137] The terminal device includes but is not limited to a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of a terminal device 3 and does not constitute a limitation on the terminal device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include input and output devices, network access devices, buses, etc.

[0138] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0139] The memory 31 may be an internal storage unit of the terminal device 3, such as a hard disk or memory of the terminal device 3. The memory 31 may also be an external storage device of the terminal device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the roaming control device. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0140] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0141] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0142] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0143] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0144] An embodiment of the present invention provides a computer program product. When the computer program product runs on a mobile terminal, the mobile terminal can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0145] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0146] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0147] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0148] In the embodiments provided by the present invention, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0149] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed over multiple network units.

[0150] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0151] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0152] As used in the present specification and the appended claims, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to monitoring, depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is monitored" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is monitored" or "in response to monitoring [described condition or event]", depending on the context.

[0153] In addition, in the description of the present specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0154] References to "one embodiment" or "some embodiments" etc. described in the present specification mean that one or more embodiments of the present invention include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0155] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A digital intelligent sanitation data optimization storage method, characterized in that: The digital intelligent sanitation data optimization storage method includes: The first calculation step: calculating feature data corresponding to the sample sanitation data, and calculating a first similarity between each of the feature data; the sample sanitation data is used to train a plurality of cluster centers; The second calculation step: calculating the conversion probability distribution between each feature data according to the first similarity; the conversion probability distribution refers to the probability of mutual conversion between two feature data; The first construction step: constructing a probability matrix according to the conversion probability distribution; wherein each column element is a conversion probability distribution between a single feature data and a plurality of other feature data; The third calculation step: performing eigenvalue decomposition on the probability matrix to obtain a probability matrix after dimensionality reduction; The second construction step: constructing a probability distribution graph based on the probability matrix; the probability distribution graph is composed of nodes and edges between nodes, the nodes are composed of the feature data, and the edges are composed of the conversion probability distribution; Extraction step: extracting clusters in the probability distribution graph and calculating cluster centers of the clusters; all nodes in the clusters are connected to one or more nodes in the clusters; Storage step: using the cluster center of the cluster cluster as an index structure, and storing the to-be-stored sanitation data corresponding to each cluster center into the storage area corresponding to each of the index structures.

2. The digital intelligent sanitation data optimization storage method according to claim 1, characterized in that: The storing step comprises: The first calculation sub-step: calculating the second similarity between the multiple cluster centers; Determining sub-step: if the second similarity is greater than a preset similarity, the cluster centers corresponding to the second similarity are fused to obtain a fused cluster center; The first selection sub-step: using the fused cluster center and other cluster centers as the index structure respectively; the other cluster centers refer to cluster centers that have not been fused; Storage sub-step: storing the to-be-stored sanitation data corresponding to each cluster center into the respective corresponding storage areas under the index structure.

3. The digital intelligent sanitation data optimization storage method according to claim 2, characterized in that: After the storing step, the method further comprises: Second calculation sub-step: obtaining a query statement or a query condition, and calculating a feature vector corresponding to the query statement or the query condition; The third calculation sub-step: calculating the third similarity between the feature vector and the fusion cluster center and other cluster centers in the index structure; Second selection sub-step: taking the storage area corresponding to the index structure of the third similarity as the target storage area; Retrieval sub-step: in the target storage area, retrieve the target sanitation data corresponding to the query statement or query condition.

4. The digital intelligent sanitation data optimization storage method according to claim 1, characterized in that: The second calculation step comprises: Substituting the first similarity into the following calculation formula, the conversion probability distribution between each feature data is obtained; Among them, P(ij) represents the conversion probability distribution between the i-th feature data and the j-th feature data, x i and x j Represents two feature data, ||x i -x j || represents the first similarity between two feature data, ∑k≠i represents the similarity between two feature data x i Sum all feature data except x k represents the kth feature data among all feature data, represents the average value of all characteristic data, and σ represents the adjustment coefficient.

5. The digital intelligent sanitation data optimization storage method according to claim 1, characterized in that: The third calculation step comprises: Fourth calculation sub-step: calculating the covariance matrix corresponding to the probability matrix; The fifth calculation sub-step: performing eigenvalue decomposition on the covariance matrix to obtain a plurality of eigenvectors and a plurality of eigenvalues; Arrangement sub-step: arranging the plurality of eigenvalues ​​and selecting eigenvectors corresponding to the first k eigenvalues; The first construction sub-step: constructing the feature vectors into a projection matrix in order; The sixth calculation sub-step: multiplying the probability matrix by the projection matrix to obtain a probability matrix after dimensionality reduction.

6. The digital intelligent sanitation data optimization storage method according to claim 1, characterized in that: The extraction step comprises: Selection sub-step: select any node in the distribution graph and obtain the current node connected to the node; The first acquisition sub-step: acquiring subsequent nodes connected to the current node; Repeat the sub-steps: taking the subsequent node as the current node, and repeatedly performing the step of obtaining the subsequent node connected to the current node until there is no new node connected, thereby obtaining the cluster; The seventh calculation sub-step: calculating the mean of the characteristic data in the cluster to obtain the cluster center.

7. The digital intelligent sanitation data optimization storage method according to claim 1, characterized in that: The second construction step comprises: The second acquisition sub-step: obtaining the value of each element in the probability matrix after dimensionality reduction and the feature data corresponding to each element, and constructing an initial distribution graph based on the value of each element and the two feature data corresponding to each element; the nodes of the distribution graph are the feature data, and the edges between the nodes are the values ​​corresponding to the two feature data; Screening sub-step: removing edges whose values ​​in the initial distribution graph are lower than a threshold, and removing isolated nodes in the initial distribution graph, to obtain the probability distribution graph.

8. A digital intelligent sanitation data optimization storage device, characterized in that: The digital intelligent sanitation data optimization storage device comprises: A first calculation unit is used to calculate feature data corresponding to the sample sanitation data and calculate a first similarity between each of the feature data; the sample sanitation data is used to train a plurality of cluster centers; A second calculation unit is used to calculate the conversion probability distribution between each feature data according to the first similarity; the conversion probability distribution refers to the probability of mutual conversion between two feature data; A first construction unit is used to construct a probability matrix according to the conversion probability distribution, wherein each column element is a conversion probability distribution between a single feature data and a plurality of other feature data; A third computing unit is used to perform eigenvalue decomposition on the probability matrix to obtain a probability matrix after dimensionality reduction; A second construction unit is used to construct a probability distribution graph based on the probability matrix; the probability distribution graph is composed of nodes and edges between nodes, the nodes are composed of the feature data, and the edges are composed of the conversion probability distribution; An extraction unit, used to extract clusters in the probability distribution graph and calculate cluster centers of the clusters; all nodes in the clusters are connected to one or more nodes in the clusters; The storage unit is used to use the cluster center of the cluster cluster as an index structure, and store the sanitation data to be stored corresponding to each cluster center into the storage area corresponding to each other under the index structure.

9. A terminal device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the digital intelligent sanitation data optimization storage method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images