45results about How to "Clustering fast" patented technology

The invention relates to a clustering method and system of a parallelized self-organizing mapping neural network based on a graphic processing unit. Compared with the traditional serialized clustering method, the invention can realize large-scale data clustering in a faster manner by parallelization of an algorithm and a parallel processing system of the graphic processing unit. The invention mainly relates to two aspects of contents: (1) firstly, designing the clustering method of the parallelized self-organizing mapping neural network according to the characteristic of high parallelized calculating capability of the graphic processing unit, wherein the method comprises the following steps of obtaining a word-frequency matrix by carrying out parallelized statistics on the word frequency of keywords in a document, calculating feature vectors of a text by parallelization to generate a feature matrix of data sets, and obtaining a cluster structure of massive data objects by the parallelized self-organizing mapping neural network; and (2) secondly, designing a parallelized text clustering system based on a CPU / GPU cooperation framework by utilizing the complementarity of the calculating capability between the graphic processing unit (GPU) and the central processing unit (CPU).

The invention discloses a system and a method for parallel video copy detection. The method includes the steps of 1, selecting key-frames of a query video and a reference video by a parallel method and extracting MIFT features of the key-frames; 2, clustering extracted feature data of the reference video by a parallel hierarchical clustering method; 3, quantizing the features of the query video and the reference video by a quantitative method according to clustered results; 4, establishing indexes of quantized data of the reference video; and 5, retrieving by the parallel method, preliminarily searching in the indexes to obtain an alternative video by utilizing quantized data of the query video and then computing space consistency and time consistency to finally confirm a copy video. According to the system for parallel video copy detection, a parallel mechanism is adopted on the basis of fast retrieval of visual vocabulary bag model BOF (beginning of file), and accordingly detection efficiency of the system is improved greatly.

The invention relates to a method for optimizing the proportion of high-proportion renewable energy under the cooperative access of transmission and distribution network, which comprises the followingsteps: 1) acquiring basic data; 2) clustering based on that basic data to obtain a typical plan scene; 3) for that typical plan scenario, establishing a collaborative planning model of transmission and distribution network, The objective of the collaborative planning model is to minimize the cost. The constraints include the independent constraints of the transmission and distribution network andthe collaborative constraints of the transmission and distribution network. The decision variables include the investment capacity of renewable energy units in the transmission and distribution network, the power generated by conventional units, the discarded energy power of renewable energy sources and the load shedding power of renewable energy units. 4) solving that transmission and distribution network collaborative planning model to obtain the optimal renewable energy investment capacity scheme; 5) obtaining the optimal distribution ratio of the high-proportion renewable energy in the transmission and distribution network based on the optimal renewable energy investment capacity scheme. Compared with the prior art, the invention has the advantage of high accuracy, strong practicability and high efficiency.

The invention discloses a text clustering method. The method comprises the steps of receiving a text clustering instruction sent by a user; pre-training a pre-determined initial language model by utilizing the to-be-clustered corpus to obtain a target language model; sequentially inputting each text in the to-be-clustered corpus into the target language model for feature extraction, obtaining a sentence vector of each text in the to-be-clustered corpus according to a model output result, and generating a to-be-clustered sentence vector set; and, by utilizing a preset clustering algorithm, clustering the to-be-clustered corpora based on the to-be-clustered sentence vector set to obtain sentence vectors corresponding to each category, and determining a clustering result of the to-be-clustered corpora. The invention further discloses an electronic device and a computer storage medium. By utilizing the method and the device, the text clustering accuracy and efficiency can be improved.

The invention discloses an optimized arrangement method for monitoring points of an urban water supply pipe network. The invention improves the traditional sensitivity analysis and cluster analysis methods separately and combines the two methods to obtain optimized distribution of monitoring points, so that the arrangement of the monitoring points can have the characteristics of being good in sensitivity, being relatively dispersed and facilitating the delimiting of abnormal regions for accurate detection, and meanwhile, the monitoring points are more widely and uniformly distributed as compared with those in the prior art, and are reasonably arranged at key positions, thereby facilitating model verification.

The invention discloses a traffic big data cleaning method based on a cloud computing framework. Firstly, the whole data source is scanned, and if missing data exist, filling is performed according to the neighbor quadratic mean value of the dimension where data of the same section are located; then, data with the similar data changing rule are clustered into one class, and the clustering center of the data of the section is obtained; finally, new data are matched with the serial number of the cluster center with the minimum distance, and updating or elimination of abnormal data is performed. Under the cloud computing (Map / Reduce of Hadoop) framework, as for the characteristics of high dimension and mass of traffic data and rapid data updating, the rapid cleaning problem of the massive traffic data is solved by the aid of the parallel computing capability of a clustering system, the similarity characteristics of the traffic data can be rapidly and effectively mined, and the method is used for cleaning abnormal data.

The invention discloses an active learning-based image detection method for a liver injury type. The method comprises the specific steps of 1, locating and extracting a liver target region from an abdominal medical image set by using a liver active shape model; 2, defining and extracting direction mode eigenvectors and grayscale range change mode eigenvectors of pixel points in the liver target region; 3, performing liver pixel density clustering center definition and clustering division based on a two-dimensional space of local density of the pixel points in the liver target region and a minimum distance from one pixel point to another pixel point with the higher density; and 4, performing image detection of the liver injury type based on an active learning method of liver pixel density clustering edge, clustering center and clustering division.

The invention discloses a traffic jam point discovery method based on Spark cluster parallel computing. The invention discloses a trajectory big data mining method, relates to the technical field of trajectory big data mining, solves the technical problem of how to quickly cluster massive trajectory data and find a traffic congestion source of an urban hot region, and comprises the following steps: (1) preprocessing the massive data, including complementing errors between trajectory points and eliminating trajectory point redundancy in a certain region for a period of time; (2) adopting a grid-means clustering algorithm to carry out clustering to obtain a plurality of target data samples; (3) clustering by adopting a neighborhood maximum density grid clustering algorithm to obtain an urbanhot traffic road network model; and (4) calculating the adjacent grid density difference by adopting a neighborhood density difference algorithm to obtain a traffic jam source, and storing the obtained result in a Spark memory. According to the method, mass data can be quickly clustered to obtain an urban traffic network model, and quick discovery of a traffic jam source region is realized.

The invention relates to the technical field of Internet, and discloses a method, a system and a server for fast clustering advertising orders. The comprises the following steps: extracting scenic global semantic features of advertising images from the advertising images included in the advertising orders; according to the scenic global semantic features of the advertising images, acquiring image fingerprints of the advertising images; according to the acquired image fingerprints of the advertising images included in any two advertising orders, calculating similarity between the any two advertising orders; judging whether the similarity between the any two advertising orders surpasses a preset similarity threshold, and if so, storing the any two advertising orders into an advertising order set corresponding to the same clustering identifier in a database. Through the adoption of the method, the system and the server, fast clustering of different advertising orders including identical or extremely similar advertising images can be achieved, so as to improve the advertising effect of the advertising orders.

The invention relates to a video abstract generation method and device, electronic equipment and a storage medium, and belongs to the technical field of the Internet. The method comprises the steps ofobtaining multiple frames of images in a target video, inputting the multiple frames of images into an image clustering model, dividing the multiple frames of images into multiple categories based onthe image clustering model, selecting the target image from each category, and splicing the selected multiple target images to obtain a video abstract of the target video. The method is based on theimage clustering model, can quickly cluster multiple frames of images, does not need to analyze the motion trail of the object in each frame of image, shortens the processing time, and improves the efficiency of generating the video abstract. Moreover, the target image is selected from each category, the target image can represent the content of the multiple frames of images in the category, the content of the target video can be accurately summarized according to the video abstract generated by the multiple target images, and the accuracy of the generated video abstract is improved.

The invention discloses a Laplacian centrality-based peak clustering method. The method comprises the steps of firstly converting a to-be-classified data set into a weighted complete graph, taking each data point as a node, and taking a distance between two data points as a weight value of an edge between the corresponding two nodes; calculating and evaluating local importance of each node of a network by using Laplacian centrality; obtaining cluster centers, wherein the cluster centers are nodes with local density centers with higher Laplacian centrality in comparison with neighbor nodes around, and relatively long distances from nodes with higher Laplacian centrality; and applying class cluster tags to all the class cluster centers, attributing a node free of the class cluster tag to a class cluster which a node with a shortest distance and the class cluster tag belongs to, and obtaining the tag of the class cluster. According to the method, an algorithm for evaluating the importance of the nodes of the network is introduced in the aspect of evaluating the importance of the to-be-classified data points; the algorithm accuracy is high; and parameter-free quick clustering can be realized.

The invention relates to the field of computers, in particular to an electric power data classification method and system based on a naive Bayesian algorithm, and the method comprises the steps: S1, obtaining data from an electric power system of an electric power company, and generating a data set; S2, taking a data subset from the data set, and carrying out incremental training to obtain the data subset; S3, calculating the frequency of each type of Ck in the data subset; S4, dividing the data subset into K sub-data subsets, and calculating the probability that the jth feature Xj is equal toajl; S5, calculating the posterior probability of each category Ck, wherein the category with the maximum probability value is the prediction category of the to-be-predicted sample; S6, removing thecurrent data subset from the data set, judging whether the data set is empty or not, if not, executing the step S2, and if yes, ending classification. According to the method, maximum likelihood estimation is adopted to represent the probabilities of various classifications for various features, and then the category with the maximum probability value is selected as the prediction category of theto-be-predicted sample, so that data classification can be quickly and accurately realized.

The embodiment of the invention provides a live broadcast room recommendation method and a related device, which are used for quickly recommending a live broadcast room to a user. The method comprisesthe steps: obtaining a user watching sequence of a live broadcast platform in a preset time period; determining a transfer weight between live broadcast rooms in the live broadcast platform accordingto the user watching sequence; determining a first directed graph of the live broadcast platform according to the transfer weight between the live broadcast rooms in the live broadcast platform; cutting the first directed graph based on a preset out-degree and / or a preset in-degree to obtain a second directed graph; calculating the transition probability of each node in the second directed graph;calculating the target arrival probability of each node in the second directed graph according to the transition probability of each node in the second directed graph; determining live broadcast roomclustering of the live broadcast platform in a preset time period based on the target arrival probability of each node in the second directed graph; and recommending a live broadcast room to the target user according to a set of live broadcast rooms watched by the target user in the preset time period and the live broadcast room clustering of the live broadcast platform.

The invention relates to a marine route network generation method and device and a computer storage medium, and the method comprises the following steps: obtaining the ship trajectory of each ship in an observation water area in a certain period of time; extracting feature points in the ship trajectory to generate a feature trajectory; performing density clustering on each feature point to obtain a plurality of clustering clusters and corresponding clustering center points; taking the clustering center point as a marine route point, replacing each feature point in the feature track with the marine route point of the corresponding clustering cluster, and generating a marine route track; and sequentially connecting the marine route points in the marine route track to generate themarine route network. The marine route network is constructed based on the ship trajectory, and the construction precision is high.

According to the DBSCAN algorithm shore bridge state classification method based on principal component analysis, sensors are installed at all positions of a shore bridge, and data transmitted by thesensors are extracted at set time intervals; Through principal component analysis, after data centralization, a characteristic covariance matrix and characteristic values and characteristic vectors thereof are obtained, the contribution degree of each component is calculated, and a plurality of previous-order principal components are taken to carry out matrix transformation; And initializing a density threshold value and a density radius for data point clustering, realizing shore bridge state classification according to a clustering result, and realizing shore bridge state monitoring. Accurateand rapid clustering of the quay crane state is achieved, non-circular domain distribution data can be clustered, a good effect is achieved, compared with a common DBSCAN algorithm, time complexity is reduced, clustering efficiency and accuracy are improved, and abnormal data can be well recognized.

The invention discloses a face image processing method and system, a server and a readable storage medium. The method comprises the steps of receiving the image information uploaded by a first terminal, and preprocessing the image information; extracting a face image from the image information according to the preprocessed image information; and clustering the face images, and storing a clusteringresult in a database. By preprocessing the image information, extracting the face image of the users, and clustering the face images matched with the users correspondingly, obtaining the clustering result and storing in the database, the similar or same face images can be quickly clustered through the clustering processing, and if the users need to download, the corresponding face images can be quickly obtained through the clustering result, so that the image information processing efficiency of a server is improved.

The invention discloses a gene expression data clustering method and device, computer equipment and a storage medium, and relates to the field of artificial intelligence, and the method comprises thesteps: selecting at least one candidate point from a to-be-clustered gene data set according to a preset weight algorithm, taking one candidate point as a reference point and calculating the candidatedistances between the reference point and the rest candidate points respectively; determining the main class cluster of the to-be-clustered gene data set and the clustering center of the main class cluster according to the candidate distances, so that the initial clustering center and the main class cluster can be efficiently selected from a large amount of gene expression data, and the clustering effect is improved. Meanwhile, according to the technical scheme, the clustering center and the main class cluster can be efficiently screened out, so that the server can quickly cluster the data according to the obtained clustering center and main class cluster, and the data processing efficiency is improved.

An image processing clustering method provided by the invention comprises the following steps: reducing the size of a mask image to obtain data to be clustered, and generating a first clustering dataset; performing DBSCAN clustering processing on the first clustering data set to obtain a cluster region, deleting semantic annotations in the first mask image based on the cluster region, and re-extracting data needing to be clustered of the first mask image; and combining the clustering results of the two parts to obtain a final clustering result. According to the method, rapid clustering of large-scale samples can be realized, and features of part of small-area features covered by large-range features can be re-extracted, so that the accuracy of a clustering result can be ensured. Comparedwith an existing clustering processing method, the problem that a small number of features in a small partial region range are omitted in the clustering processing process can be avoided. The invention further provides an image processing clustering system and an electronic device, and the image processing clustering system and the electronic device have the same beneficial effects as the method.

The invention belongs to the field of vehicle detection and recognition, and particularly relates to an intelligent bee colony clustering method and a vehicle target detection method. The invention relates to an intelligent bee colony clustering method, which mainly comprises the following steps of: 1) establishing a common target of a bee colony, namely searching a plurality of high-quality honeysources; 2) making a work plan; 3) establishing an assessment system; And 4) outputting a clustering center result. According to the intelligent bee colony clustering method, the bee colony algorithmand the clustering model are effectively combined, and a team management mode is integrated, so that a good clustering effect can be achieved, the calculation amount of subsequent similarity measurement is reduced, and the efficiency and the precision are improved. According to the vehicle target detection method, the clustering center is obtained after vehicle characteristics are subjected to efficient clustering analysis, the optimized bee colony algorithm is applied to the clustering analysis by fusing the thought of team management, the working mode that bee members in a team find high-quality honey sources is imitated, and the purpose of rapid clustering is achieved.

The invention provides a data processing method, electronic equipment and a storage medium, and is applied to a server, the server is connected with a plurality of cameras, the method comprises the following steps: in response to the acquisition of Lx, adding Lx to a historical track segment set to obtain an updated track segment set M = {L1, L2,..., Ln} and La = {Pa1, Pa2,..., Paf (a)}, M including Lx; according to the updated trajectory segment set M, determining a trajectory feature set N = {T1, T2,..., Tn} and Ta = Avg (ta1, ta2,..., taf (a)), where Ta is a trajectory feature corresponding to La, and tab is an object feature obtained by performing feature extraction on an object in Pab; generating a similarity matrix S according to the track feature set N; and clustering the updated trajectory segment set M according to the similarity matrix S to obtain at least one global trajectory, each global trajectory including at least one trajectory segment, and each trajectory segment in the same global trajectory corresponding to the same object.

The invention relates to a data quality governance system based on a message bus. The system performs the steps of receiving power monitoring data sent by a plurality of data sending ends through themessage bus and stored in a classified mode; determining a power monitoring data quality evaluation index, and constructing a data quality rule verification library; clustering the power monitoring data by adopting a k-means clustering method optimized on the basis of a grey wolf longhorn beard search algorithm, and dividing the power monitoring data into a plurality of data subsets; for each datasubset, carrying out accuracy evaluation through an LOF abnormal value detection algorithm, and carrying out integrity, consistency, availability and real-time evaluation through data quality rule verification; and calculating the weight of each evaluation index through an improved analytic hierarchy process, and calculating a comprehensive evaluation result of the power monitoring data quality.According to the system, the multi-dimensional quality evaluation of the power monitoring data is realized, and the reliability of the quality management of the power monitoring data is improved.

The invention relates to the field of data processing, and discloses a data auditing method in a sandbox environment, which comprises the following steps of: obtaining a user operation data set from auser system, clustering the user operation data set to obtain a check data set and a reference case set, splitting the check data set to obtain field data sets, according to the field data sets, respectively constructing corresponding gateway verification programs, respectively verifying the corresponding field data sets by utilizing the gateway verification programs, generating a case configuration file according to the reference case set when all the field data sets pass the verification, and indexing the case set from a pre-constructed rear-end case library according to the case configuration file, and when the case set satisfies a preset index relationship, distributing the case set to the user system. The invention further discloses a data auditing method system in the sandbox environment, electronic equipment and a storage medium. According to the invention, the problems of overlarge calculation pressure and low security during data auditing can be solved.

The invention relates to the field of classification, in particular to a short message clustering method based on a text structure, the method comprising the following steps: S1, preprocessing word segmentation of short messages in a short message set; S2, calculating similarity degree based on short message structure for any two short messages pretreated by word segmentation, and constructing similarity degree matrix; S3, carrying out a clustering process, comprising: comparing each value in the similarity matrix with a preset similarity threshold value respectively; when the similarity valueis greater than the similarity threshold value, classifying two short messages corresponding to the similarity value into the same category, and finally obtaining a short message clustering result.

The invention discloses an optimal layout method for urban water supply pipe network monitoring points. The present invention improves the traditional sensitivity analysis and cluster analysis methods respectively, and combines the two methods to obtain an optimized measuring point distribution, so that the measuring point arrangement not only has good sensitivity, but also is relatively dispersed, which is beneficial to demarcate abnormal areas for accurate detection At the same time, the distribution of measuring points is wider and more uniform than the existing measuring points, and the layout of key positions is reasonable, which is conducive to model verification.

The invention relates to a rod-shaped object regular three-dimensional modeling method and rod-shaped object regular three-dimensional modeling system based on density peak clustering. The rod-shaped object regular three-dimensional modeling method comprises the steps of performing voxel resampling on original point cloud data which are acquired by an on-vehicle laser radar, eliminating outliers, performing mesh segmentation on a scene point cloud, performing ground point elimination and high-rise point elimination on each grid through elevation straight-through filtering, and respectively performing projection on three coordinate planes of a three-dimensional coordinate system; respectively clustering in three projection planes on each grid in a clustering manner based on a density peak and a distance attribute; limiting the spatial range of the point clouds in a same kind by a regular spatial cubic bounding box, extracting the rod-shaped object in the spatial cubic bounding box according to priori knowledge, and obtaining a rod-shaped member characteristic parameter; and performing modeling for restoring the rod-shaped object. The rod-shaped object regular three-dimensional modeling method and the rod-shaped object regular three-dimensional modeling system can perform quick, efficient and accurate extraction on rod-shaped objects in a majority of environments and integrate with a digital city, thereby realizing fine and true modeling on the rod-shaped objects in the digital city.

The invention relates to a data quality management system based on a message bus. The invention receives and stores power monitoring data sent by a plurality of data sending ends through the message bus; determines power monitoring data quality evaluation indicators, and builds a data quality rule verification library; The k-means clustering method based on the gray wolf-beetle whisker search algorithm is used to cluster the power monitoring data, and the power monitoring data is divided into multiple data subsets; for each data subset, LOF outlier detection is performed The accuracy is evaluated by the algorithm, and the integrity, consistency, usability, and real-time evaluation are performed through the verification of the data quality rules; the weight of each evaluation index is calculated by the improved analytic hierarchy process, and the comprehensive evaluation result of the power monitoring data quality is calculated. The invention realizes the multi-dimensional quality evaluation of the electric power monitoring data, and improves the reliability of the quality management of the electric power monitoring data.

The invention provides a data stream clustering method and device based on a density peak value, and relates to the technical field of data processing. According to the method, a Jaccard similarity distance and a Gaussian kernel function are combined to calculate the local density of an evolved data stream, a heuristic strategy for kernel density estimation is introduced, and a local clustering center point is selected from points with large local density. And finally, local clustering center points are recorded through a variable-scale bucket sequence, and when a clustering request arrives, the local clustering centers are merged to obtain a global clustering center. According to the method, the concept drift in the data stream can be identified, the data stream is quickly clustered, and good clustering performance is achieved.

Embodiments of the present invention provide a live broadcast room recommendation method and related equipment, which are used to quickly recommend live broadcast rooms to users. The method includes: obtaining the user viewing sequence of the live broadcast platform within a preset time period; determining the transfer weight between live broadcast rooms on the live broadcast platform according to the user viewing sequence; determining the first live broadcast platform according to the transfer weight between live broadcast rooms on the live broadcast platform A directed graph; clipping the first directed graph based on the preset out-degree and / or preset in-degree to obtain a second directed graph; calculating the transition probability of each node in the second directed graph; according to the first The transition probability of each node in the second directed graph calculates the target attainment probability of each node in the second directed graph; based on the target attainment probability of each node in the second directed graph, determine the live broadcast platform in the preset time period. category; recommend live rooms to target users based on the collection of live rooms watched by target users within a preset time period and the clustering of live rooms on the live broadcast platform.

The invention discloses a real-time image clustering method, comprising the following steps: A. batch input images, and obtain image global descriptors through edge feature extraction; B. obtain image local feature descriptors through low-resolution image feature extraction; C. Input the image global descriptor and the image local feature descriptor; D. Create a three-layer self-organizing mapping neural network, and use the image global descriptor to select the first N competing layer neurons to enter the activation state; E. Calculate the image local feature descriptor and each The distance between each nerve ending under each activation neuron is obtained, and the activated neuron that responds successfully as a whole is obtained; F. According to the number of activated neurons that respond successfully as a whole, the clustering of images and the learning or combination of activated neurons are performed, or Create new competition layer neurons using image global descriptors and image local feature descriptors. The present invention improves the accuracy of the image clustering result and the real-time performance and stability of the clustering process as a whole.

The invention provides an electric vehicle charging load characteristic parameter recognition method and device based on a complex coupling network multi-agent technology, equipment and a medium. The method comprises the following steps: S1, designing a consistency control method, introducing complex coupling network containment consistency control into data clustering analysis, and quickly solving an initial clustering center; s2, a bipartite k-means method is adopted to quickly identify an expected value of an electric vehicle initial charging moment in a typical scene, and an electric vehicle charging probability and a charging initial time probability distribution function are extracted; compared with the prior art, the technical problems that an existing electric vehicle charging load modeling method based on data driving is complex in operation and difficult to achieve engineering application are solved, human intervention is not needed, initial clustering center selection is automatically completed, and clustering precision is improved; the charging probability and charging time period probability distribution function of the electric vehicle can be quickly and accurately solved, and the method has the characteristics of simple calculation and quick clustering.