71 results about "Unsupervised algorithm" patented technology

A system and a method are disclosed for clustering images of objects seen from different viewpoints. That is, given an unlabelled set of images of n objects, an unsupervised algorithm groups the images into N disjoint subsets such that each subset only contains images of a single object. The clustering method makes use of a broad geometric framework that exploits the interplay between the geometry of appearance manifolds and the symmetry of the 2D affine group.

The invention discloses an automatic marking method for a natural scene image, and belongs to the field of computer vision. The method comprises the following steps that image features are extracted; an original image is segmented by adopting an unsupervised algorithm so that a super-pixel graph is generated; modeling of a pixel marking model is performed through CRF and significant prior information is embedded in the model; and the model is solved and pixel marking is realized. The CRF is adopted to act as a basic model, the significant detection prior information is introduced in the CRF model, and separation of a foreground target and a background can be realized through significant detection and a universal connection association relation between the super-pixels is constructed in a foreground target area. The significant detection prior information is introduced so that the classification precision of the foreground target in the image can be effectively enhanced. Meanwhile, the problem of classification "crosstalk" of the foreground and the background can be effectively solved by the separation of the foreground area and the background area. Therefore, the overall classification precision of pixel marking can be effectively enhanced by the method, and the method has substantial effect for the scenes of relatively complex foreground target profiles and subareas of highly different colors and textures.

The invention relates to a biomedicine technology subject mining method based on LDA, belonging to the technical field of information retrieval. The method of the present invention first employs an LDA to view a document as a combination of vectors of a contained word, then the number of semantic topics K is determined by using the evaluation function Perplexity; and finally, a probability p of each document di on all Topics is calculated, and two matrices, one doc-Topic matrix and the other word-Topic matrix, are obtained, so that LDA projects documents and words onto a set of Topics, tryingto find out the potential relationship between documents and words, documents and documents, words and words. LDA is an unsupervised algorithm, and every Topic does not require a specified condition.However, after clustering, the probability distribution of the words on each Topic is calculated, and the words with high probability on the Topic can describe the meaning of the Topic very well.

The invention discloses a fault root cause positioning method for a micro-service system, which comprises the following steps of: acquiring call chain data in the micro-service system in real time, converting the call chain data into four business indexes and monitoring in real time, when an exception is found, constructing a service topological graph, a process topological graph and a host topological graph by using a call chain in an exception time period, calculating an abnormal score of each node on the topological graph, and finally positioning to a root cause node, namely a process node or a host node, according to a sequence from large depth to small depth and from process to host. According to the obtained call chain data, the topological relation of the host level can be dynamically constructed, the topological relation of the process level and the topological relation of the service level can also be dynamically constructed, and data guarantee is provided for more accurate root cause positioning. According to the method, the call chain data in the abnormal time period are analyzed in real time by utilizing an unsupervised algorithm, and training data and labels are not needed.

The embodiment of the invention provides a hidden feature extraction method and device, computer equipment and a computer readable storage medium. The embodiment of the invention belongs to the technical field of text classification. According to the embodiment of the invention, hidden features are extracted; The method comprises the steps of obtaining a first corpus for performing hidden featureextraction, performing word embedding on the first corpus to convert the first corpus into a word vector, extracting word vector characteristics of the word vectors through a convolutional neural network; clustering description is carried out on the word vectors by adopting an unsupervised algorithm; secondly, encoding the word vector characteristics in a self-encoding mode to extract hidden characteristics of the word vector characteristics so as to realize dimension reduction processing on the data of the word vector characteristics; Therefore, hidden features of the corpora are extracted through unsupervised learning, the precision of subsequent learning modeling can be improved, and the influence of the training data amount is overcome.

The embodiment of the invention provides a card raising number detection method and system based on an unsupervised algorithm. The method comprises the following steps: 1) collecting operator electriccanal login log data; 2) acquiring login behavior characteristics of the user from the login log data, taking the login behavior characteristics of the user as a first characteristic set, and takinghigh-dimensional statistical characteristics corresponding to the login behavior characteristics of the user as a second characteristic set; 3) identifying each abnormal group corresponding to the first feature set by using an isolated forest algorithm; clustering the features in the second feature set by using a clustering algorithm to obtain a plurality of clusters, and obtaining abnormal clusters according to the stability of the login behavior features; and 4) determining whether the number corresponding to the abnormal group belongs to the card raising number or not according to the number of the numbers clustered into the abnormal cluster in the numbers corresponding to the abnormal group and the proportion of the numbers corresponding to the abnormal group. By applying the embodiment of the invention, the identification accuracy of the card raising number can be improved.

A method for generating an adaptive security model includes: generating an initial security model with respect to data input via an Internet during a learning process; and continuously updating the initial security model by applying characteristics of the input data during an online process. Said generating an initial security model includes: matching the input data with a unit having a weight vector with distance closest to the input data using a first unsupervised algorithm; generating a map composed of weight vectors of units; and performing a second unsupervised algorithm using the weight vectors forming the map as input values to partition an attack cluster.

Methods, apparatus, and systems for analyzing data trends are described herein. The present disclosure includes the identification of trending terms in data through the use of an unsupervised algorithm. Trends are identified and counted during a first and second time period. The frequency and co-occurrence of groups of terms is compared to determine a set of trending terms without reference to a library of pre-defined terms. The set of trending terms is displayed to a user.

The invention discloses a traffic identification method and device, an electronic device and a storage medium. The method comprises the following steps of monitoring the flow of a target service, andcalculating the flow mean value ratio of the target service; if the calculated traffic mean value ratio exceeds a first threshold value, determining a traffic sudden increase moment; obtaining the user data corresponding to the traffic sudden increase moment, and screening out abnormal user data in the user data based on a graph semi-supervised method; and determining an abnormal user according tothe abnormal user data, and identifying the flow of the abnormal user as abnormal flow. According to the technical scheme, a mean shift thought is adopted to the monitor traffic sudden increase, theabnormal traffic generation is determined, a graph semi-supervised learning algorithm can represent the similarity of access behaviors which are not intercepted, the interpretability is high, the requirement for data is lower than that of an unsupervised algorithm, and an obtained result is more stable; and the omission is avoided through a mode of firstly determining the abnormal users and then identifying the abnormal flow.

The embodiment of the invention provides a text data viewpoint generation method and device and electronic equipment. The text data viewpoint generation method comprises the following steps: acquiringtarget text data to be processed; extracting a first viewpoint of the target text data through a supervised machine learning model, and mining a second viewpoint of the target text data based on an unsupervised algorithm; determining candidate evaluation themes and candidate viewpoint contents of the target text data according to a relation between characters contained in the first viewpoint andcharacters contained in the second viewpoint; and generating a viewpoint of the target text data according to the candidate evaluation theme and the candidate viewpoint content. According to the technical scheme, the supervised method can be fully utilized to accurately identify the viewpoints of the text data, meanwhile, the problems that the supervised method is low in recall rate and slow in iteration processing can be effectively solved through the unsupervised algorithm, and the identification accuracy and the identification efficiency of the viewpoints of the text data are improved.

The invention provides a behavior sequence anomaly detection method based on an unsupervised algorithm. The method comprises the steps: calculating the time interval of two operations based on the operation data of an enterprise web system through the sequence of user operations, and segmenting a user behavior sequence according to whether the time interval of the two operations is greater than a preset threshold or not, and training a probability suffix tree model, outputting a probability value corresponding to the user behavior sequence according to the probability suffix tree model, taking the probability value corresponding to the user as a feature, i.e., input of an isolated forest model, and judging whether the user behavior is abnormal or not according to a model output result.

A document keyword extraction method based on a BERT model comprises the following steps that each document in a document set is coded through the BERT model, and the attention weight of document semantics generated by the BERT model to each sub-word is extracted; restoring the sub-words into words, and aggregating the attention weights of the sub-words into the attention weight of the words; the attention weights of the same word at different positions in the document are aggregated into the attention weight, irrelevant to the position, of the word, and the attention weight is recorded as p (wordweight '2jeemaa2' doc); calculating the attention weight of each word on the document set, and recording the attention weight of each word on the document set as p (wordweight '2jeemaa2' corpus); and combining the p (wordweight '2jeemaa2' doc) and the p (wordweight '2jeemaa2' corpus), and selecting N words with the highest final attention weight as the keyword of the document. According to the method, the BERT model is used for extracting the document semantic representation to calculate the word attention weight distribution, the keyword extraction is finally realized, the word frequency information is considered, the problem that the semantics is ignored by the traditional unsupervised algorithm is effectively solved, and the keyword extraction accuracy and recall rate are improved.

The invention relates to a load curve data visualization method based on the combination of supervised and unsupervised algorithms, and the method comprises the steps: firstly obtaining the precise label data of a load curve through employing an unsupervised optimization spectral clustering algorithm based on the double-scale similarity measurement of distance and curve form; secondly, learning intrinsic characteristics of a large-scale to-be-classified load curve by adopting a sparse automatic encoder neural network to obtain a hidden layer weight matrix, namely an optimized initial parameterof the neural network; and finally, based on the obtained label data, training a support vector machine neural network classifier to realize supervised classification of the large-scale to-be-classified load curves. Supervised and unsupervised algorithms are combined, so that more accurate load curve category label data can be obtained, and the problem of relatively low calculation efficiency caused by big data is improved to a certain extent.

The invention relates to a data-driven unsupervised algorithm for analyzing sensor data to detect abnormal valve operation. A computer-implemented method, system, and computer program product are provided. A plurality of maintenance messages (MMSGs) are identified. Each MMSG is associated with at least one shut-off valve. A sensor parameter is identified based on an analysis of sensor parameters associated with the shut-off valves of each MMSG. A threshold value for the sensor parameter is identified as being associated with abnormal operation of the respective shut-off valves. A sensor associated with a first shut-off valve captures values for the sensor parameter during a first and second predefined time period, the first and second predefined time periods associated with an opening anda closing of the first shut-off valve. Upon determining that a difference between the maximum values of the sensor values captured during the first and second predefined time periods exceeds the firstthreshold value, a determination is made that the first shut-off valve is operating abnormally.

The invention relates to an anomaly detection method and system based on log information, and computer device. The anomaly detection method based on the log information comprises the following steps: obtaining structured data: exporting a log, extracting attribute features of the log through a regular expression, and converting the attribute features into structured data; performing unsupervised detection model training: carrying out dimensionality reduction on the structured data, carrying out data clustering on the internal structure of the structured data by utilizing a clustering algorithm, and repeating the step to obtain an unsupervised recognition model; performing supervised detection model training: constructing time sequence feature data by using a timestamp according to the structured data, and training a supervised recognition model based on the time sequence feature data; and performing anomaly detection: importing a to-be-detected log into the unsupervised recognition model and the supervised recognition model, and performing anomaly detection. By using a supervised algorithm and an unsupervised algorithm, abnormal log recognition is carried out from different angles, and the log anomaly detection effect is greatly improved.

The invention provides an operation and maintenance data feature selection method and device, and the method comprises the steps: obtaining an original data sample; preprocessing the original data sample to obtain a multi-dimensional data sample; calculating the multi-dimensional data sample through a preset algorithm, and when the calculated value of the cost expression is minimum, outputting the feature weight of each dimension of data; and screening out a target data set from the multi-dimensional data sample according to the feature weight of each dimension of data and a preset weight threshold. Therefore, a feature selection method capable of adapting to an actual operation and maintenance environment is provided, the method does not depend on experience of operation and maintenance personnel, a large amount of historical data and manual annotation, and does not depend on one algorithm to detect the self effect, so that the method can adapt to various downstream early warning algorithms or analysis algorithms, and combines the advantages of a supervised algorithm and an unsupervised algorithm; the method not only can learn the characteristics of historical faults and position the high-frequency abnormal dimensions, but also can effectively judge the dimensions without faults in history.

Raw data is received from an industrial machine. The industrial machine includes one or more sensors that obtain the data, and the sensors transmit the raw data to a central processing center. The raw data is received at the central processing center and an unsupervised kernel-based algorithm is recursively applied to the raw data. The application of the unsupervised kernel-based algorithm is effective to learn characteristics of the raw data and to determine from the raw data a class of acceptable data. The class of acceptable data is data having a degree of confidence above a predetermined level that the data was obtained during a healthy operation of the machine. The acceptable data is successively determined and refined upon each application of the unsupervised kernel-based algorithm. The unsupervised kernel-based algorithm is executed until a condition is met.

The invention belongs to the technical field of flow control, and discloses a network protocol flow control system, which comprises a data packet detection module, a virus detection module, a flow monitoring module, a main control module, an address identification module, a transmission control module, a flow filtering module, a flow classification module, a caching module and a display module. According to the invention, the network address to be filtered can be matched with the node in the binary tree filtering model for less times through the flow filtering module to determine whether the network address to be filtered is a known network address or not, and then the network address to be filtered is filtered. The storage analysis data volume of the network traffic is greatly reduced, sothat rapid filtering of the network traffic is realized; classifying by combining an unsupervised algorithm and a supervised algorithm in a machine learning method through a flow classification module; the combination of the two can reduce the system time and memory overhead on the premise of ensuring higher classification accuracy, and improve the classification efficiency.

The invention discloses a maintenance shop rating method, system, electronic device and storage medium. That method includes: acquiring characteristic data of the maintenance shop; performing K-meansclustering operation on all the characteristic data to obtain K class families, wherein K is a positive integer; the mapping relationship between the K class families and the K grades is established,and the repair shop rating result is obtained according to the K grades. The maintenance shop rating method provided by the present application characterizes the maintenance ability of the maintenanceshop through the characteristic data, and performs K-means clustering operation on all the characteristic data to obtain K class families, corresponding to K ranks of evaluation results. In the rating process of the maintenance shop, the unsupervised algorithm is implemented to avoid the inaccurate rating results caused by human subjective factors.

The invention relates to an AI-based abnormal crowd identification method, which comprises the following steps: 1, collecting information, namely collecting all equipment information of equipment appearing on a platform by taking the equipment as a research object; 2, implementing an unsupervised algorithm, and representing the collected data by using a data structure diagram; and 3, implementinga supervised algorithm, and constructing a binary classification prediction model through the data structure diagram in the step 2. The method solves the technical problems that the prior art is obviously lagging since users need to have actual batch abnormal behaviors, it takes a long time to keep and maintain an account, simulation is high, and the dark industry makes profits at one night and then disappears.

The invention belongs to the field of data processing, and discloses a network service abnormal data detection method and device, computer equipment and a readable storage medium. The method comprisesthe following steps: pre-judging an unsupervised algorithm for judging data exception; dividing an unsupervised algorithm; reducing the data calculation amount of preliminary detection; judging a supervised algorithm through a preset anomaly detection model to generate a target judgment result which comprises a judgment condition and a judgment confidence coefficient representing the credibilityof the judgment condition. According to the method, the same data is detected through multiple algorithms, and the technical problem that in the prior art, data exception judgment is conducted througha single algorithm, and consequently the accuracy of data exception judgment is low is solved.