30 results about "Algorithm parallelization" patented technology

The invention discloses a distributed network security event associated analysis method based on a knowledge graph. The method specifically comprises the following steps of 1, building a network security knowledge graph comprising five dimensions such as the basic dimension, the loophole dimension, the threat dimension, the alarm event dimension and the attack rule dimension; 2, designing a security event implementation associated analysis algorithm based on the knowledge graph built in the step 1; and 3, building a real-time big data analysis platform, applying the associated analysis algorithm designed in the step 2 to the built big data platform, and thus achieving a distributed associated analysis system. According to the method provided by the invention, related technologies of current big data process are fully used for dealing with the large data volume, the associated analysis algorithm is parallelized, and the design of the distributed associated analysis algorithm based on the knowledge graph is achieved.

The invention discloses a matrix decomposition parallelization method based on a graph calculation model. Matrix decomposition can be flexibly brought into more user information. The matrix decomposition deduces the hidden semantic vectors of a user and an article according to the score of the article by the user, and then, recommendation is carried out according to the hidden semantic vectors of the user and the article. However, in a practical application scene, the implementation of a matrix decomposition recommendation algorithm needs to consume a great quantity of time, and traditional commercial requirements can not be met. A distributed calculation platform can be used for carrying out parallelization on the matrix decomposition recommendation algorithm to effectively solve the problem, and meanwhile, a multiple-iteration calculation problem is in the presence in the implementation of the matrix decomposition recommendation algorithm. The invention puts forward the Spark-based GraphX graph calculation frame to realize matrix decomposition parallelization. Compared with a traditional MapReduce calculation graph model, the graph calculation frame has the obvious advantages on the aspect of the solving of multiple-iteration problems and execution efficiency.

The invention belongs to the intersection field of natural language processing and machine learning, and provides a parallel k-means algorithm used for high-dimensional text data. According to the algorithm, firstly, the text data are vectorized, then a dimension reduction model is established for a matrix after vectorization, and the high-dimensional data are converted into low-dimensional data with highly efficient features; and then algorithm accuracy is further improved through the optimized k-means clustering algorithm, then clustering is carried out on the data after dimension reduction,algorithm parallelization is realized through GPU and MPI technology, and finally, improvement of accuracy and efficiency of the high-dimensional text data is realized. The algorithm can greatly improve accuracy of text clustering, and improve running speed and portability of the algorithm.

The invention discloses an optimization method for mapping virtual machines to physical machines. The method comprises the following steps of S1, modeling virtual machine demands; S2, determining a threshold; S3, establishing a priority queue; S4, carrying out batch dequeuing; S5, designing an optimization kernel; and S6, carrying out algorithm parallelization. According to the optimization methodfor mapping the virtual machines to the physical machines provided by the invention, the validity and low delay property of the algorithm are improved. According to the method, allocation positions of the virtual machines can be timely and effectively determined according to the specific virtual machine demands; and when the virtual demands are dynamically changed, the targeted dynamic adjustmentis carried out for the mapping from virtual machine resources to physical machine resources. In an application process, a utilization rate of cloud resources can be improved, and the operation cost of a cloud service provider is reduced.

The invention discloses a blogRank algorithm parallelization processing construction method based on Haloop. Blog data are preprocessed; every iterative process of the algorithm is abstracted into a MapReduce model, and the model is composed of two concrete MapReduce processes; cyclic invariables and cyclic variables in the iterative process are separated; appropriate iteration end conditions and the maximum iteration times are set; calculation is performed with a programmatic interface provided by a Haloop frame. After the test, under the condition of a large data volume, compared with a traditional one-machine computing method applying the matrix and a distributed computing method applying a Hadoop frame, the construction method applying the Haloop frame obviously promotes operating efficiency, and the larger the data volume is, the more the efficiency is promoted. The method can effectively reduces the effect on executing efficiency of the BlogRank algorithm caused by iteration, and can well adapt to requirements for processing a large volume of data with the algorithm.

The invention relates to a load reduction parallel computing method which includes the following steps: reading branch node breaking data, equally grouping the branch node breaking data into multiplepartitions of a data parallel computing platform, and acquiring a load reduction algorithm package programmed by Matlab and Java; and calling a load reduction algorithm in each partition to compute branch node breaking data in the partition and acquire branch node load reduction. By calling a load reduction algorithm package programmed by Matlab and Java to compute branch node breaking data in parallel and acquire branch node load reduction, the difficulty and complexity of load reduction algorithm parallel development are greatly reduced. When a load reduction algorithm is newly added, the new algorithm can be parallelized as long as Matlab and Java mixed programming is carried out on the new algorithm to get a load reduction algorithm package. The method is of high universality.

The invention provides a method for predicating performance of a technological enterprise through a CIO/CTO social network. The method mainly comprises the following steps of (1), acquiring data, acquiring Standard & Poor's data, BoardEx data, etc; (2), performing original data preprocessing, namely processing the acquired enterprise data; (3), executing a weighted central algorithm, namely presenting a weighted centrality algorithm in consideration of common influence of multiple centralities, and replacing a single centrality by the weighted centrality; (4), executing a CIO/CTO value evaluating method, evaluating the CIO/CTO value by means of the centrality, and analyzing network centrality of the CIO/CTO; (5), performing algorithm parallelization, and improving data processing efficiency by means of parallel operation capability of a Spark cluster; and (6), performing data analysis, and evaluating the performance of the enterprise through weighted centrality value estimation by means of Probit regression analysis and an OLS model. According to the method of the invention, performance of the enterprise is predicated by means of the weighted centrality, thereby settling a problem of insufficient research to the complicated social network on the condition that only a single measurement standard is considered.

The invention belongs to the technical field of industrial big data, and particularly relates to an improved product quality abnormal data FP-Growth correlation analysis method. The main content of the method of the invention comprises the product quality abnormal data association analysis characterized in that the multi-factor association analysis is carried out on the product production qualityand the production process data, a series of association rules are mined based on a multi-factor association analysis algorithm, the potential problems of some indexes in the quality data are found, and the factors causing product quality abnormal influences are conveniently positioned; the FP-Tree data structure improvement characterized in that a field tail _ Link is newly added on the basis ofan FP-Tree frequent item header table, and the current last node of each data item is recorded, so that the repeated traversal linked list operation when a new node is inserted is avoided, and the FP-Tree tree building efficiency is improved; the improvement of the parallelization strategy of the FP-Growth association algorithm characterized in that an FP-Growth algorithm mining frequent mode is executed parallelly on each transaction set group to break through the original mode of firstly building a tree and then parallelizing, so that the calculation efficiency of each node in the parallelization execution is improved.

The invention discloses a warehouse sorting path optimization method based on improved GA-PAC, and the method comprises the steps: taking the delivery time of goods as an evaluation index according tothe setting of a roadway of a storage center, and taking the shortest sorting path as an optimization target; comparing the performances of an ant colony algorithm, a genetic algorithm, a parallelized ant colony algorithm and a warehouse sorting path optimization algorithm for genetically optimizing parallel ant colony parameters through an Oliver30 standard model; wherein the parallelized ant colony interaction method not only has the characteristic that the independent ant colonies are simple and convenient, but also makes up the limitation that no interaction exists among the ant coloniesand the information transmission direction is unidirectional. The parallelized interactive ant colony is combined and optimized by adopting a genetic algorithm through operations such as selection, crossing, variation, re-insertion, decoding and the like; the method has the advantages that the optimization capability is high, the method stability is good, a better solution is accurately, quickly and stably found out, the warehouse sorting path optimization algorithm mapped to GA-PAC through distributed calculation is higher in optimization capability, higher in algorithm stability and higher in optimization speed, and the sorting path optimization and stocking efficiency is improved.

The invention discloses an abnormity detection method and a device for parallelization of an isolated forest algorithm based on Flink. The isolated forest algorithm is realized based on the real-timeprocessing platform Flink platform, the calculation speed of the isolated forest algorithm is greatly improved, and the problem that the exception detection is limited by the data volume when the isolated forest is used for exception detection in a stand-alone mode is solved. Transverse expansion is carried out by configuring and adding machines, and an abnormity detection task is also carried outin a real-time scene. The method is implemented by adopting the Flink, and abnormity detection is carried out on the data in real time by utilizing the real-time processing characteristic of the Flink. The parallelism degree of the algorithm can be transversely expanded by setting the parallelism degree of the operator of the Flink, so that the parallelism degree of the algorithm is not limited by the data volume when abnormity detection is carried out on mass data.

The invention relates to a parallelization signal quality evaluation method. The method comprises multithreading, loop parallelization, vector parallelization and matrix operation; the loop parallelization adopts CUDA stream to realize navigation signal capture and acceleration; during capture processing operation, loops are independent of each other, concurrent execution is carried out by flow, and concurrent operation is managed by a program through flow; an optimal parameter algorithm is searched by adopting hyper-parameter optimization reinforcement learning in the cycle of the outer-layercapture statistics times; a reduction summation algorithm is adopted for vector parallelization and matrix operation; a plurality of branches are started by multithreading to perform parallelizationfor tracking and accelerating an IQ baseband branch, wherein different branches in the CUDA can concurrently execute respective tracking through different threads; in capture, tracking, and IQ baseband branches, for variables that are always unchanged in each cycle, the variables are extracted out of the cycle for individual calculations. The method has the advantages that the operation efficiencyis high, and the requirements of GNSS signal quality evaluation for real-time performance, reliability and high throughput of data processing are met.

The invention discloses a method for mining dominant influence factors of a power utilization behavior of a user. The method comprises the steps of performing multiple correlation check on an influence factor data set; if multiple correlation exists, performing screening by adopting a stepwise regression method; judging whether a target data table has the dominant influence factors or not by applying typical correlation analysis; if the target data table has the dominant influence factors, performing clustering analysis on the target data table by adopting an improved K-center point clusteringalgorithm; and finally obtaining data about the dominant influence factors. According to the method, analysis of the power utilization behavior of the user is changed to variable analysis research from sample statistical classification, so that the influence factors of the power utilization behavior of the user can be better mined; secondly, K-center point clustering analysis is improved, namely,an ideal solution method is introduced for determining an initial clustering center, so that falling into a local optimal solution is avoided; clustering algorithm parallelization is realized, so that the data processing capability of the algorithm is remarkably improved; and finally, an output result intuitively displays spatial and temporal distribution characteristics of the dominant influencefactors in multiple forms.

The invention belongs to the field of target identification, and particularly relates to a rapid identification method based on a lightweight DAB-Net network. The method includes: taking a lightweightDAB-Net network structure as a basis, carrying out model quantification; parallelizing the algorithm by adopting multi-thread pipelined calculation; decomposing the algorithm into a plurality of tasks, allocating corresponding pairs of threads, and executing an independent task by each thread; gPU forward propagation operation time is taken as the minimum cycle period, and image reading, data input processing, data output processing and Opencv subsequent processing are included in GPU operation time consumption; awakening other waiting threads through a thread synchronization signal to achieve thread synchronization; data input at the current moment comes from data output of the previous moment and the previous state, and parallel algorithm optimization acceleration is achieved by delaying data output of four frames. According to the invention, the identification speed is improved based on the lightweight DAB-Net network.

The invention provides a network abnormal traffic analysis method and system based on Spark and clustering, and the method comprises the steps: carrying out the clustering analysis of network trafficthrough clustering through a Spark big data processing platform, and carrying out the abnormal traffic analysis of classified network traffic through a detection algorithm. On the basis of the primaryclustering, the abnormal flow cluster and the normal flow cluster are judged by utilizing the Mahalanobis distance, so that the purpose of distinguishing the normal flow from the abnormal flow is achieved. In order to further improve the efficiency of the method, a means of parallelizing the K-means algorithm is adopted in Spark-based clustering flow analysis, the calculation efficiency of the algorithm is improved through parallelization, the requirements of the algorithm for machine memory and kernel processing are reduced, and the practicability of the algorithm is improved.

The invention provides a complex network topology characteristic parameter calculation method and system based on MapReduce. An algorithm parallel method based on message transmission is employed. The method comprises the steps of S1, generating update messages; S2, transmitting the messages; and S3, updating internal state information of nodes. For the problem that the efficiency is relatively low when conventional stand-alone algorithms are used for calculating large-scale network topology characteristic parameters, the invention provides a method for transplanting the stand-alone algorithms for the network topology characteristic parameters to a MapReduce calculation framework in parallel, the problem occurred in the process of transplanting the stand-alone algorithms to the MapReduce in parallel is overcome, the network topology characteristic parameters are calculated in parallel through utilization of a Hadoop calculation platform, and the calculation efficiency of the network topology characteristic parameters is improved.

The invention discloses an SM4 encryption and decryption algorithm parallelization implementation method based on a tower domain optimization S box, and the method comprises the steps: constructing a bit matrix transpose transformation function Transs (.), outputting a transformation bit matrix of an input bit matrix, dividing the transformation bit matrix into bit square matrixes, and carrying out the transpose of bit granularity; copying and converting the encrypted bit key to obtain a round key; splitting the data of the transformed bit matrix into bit matrixes, performing iterative computation in combination with a round key, optimizing the S-box operation efficiency by using a tower domain technology, performing reverse-sequence operation after the iterative computation is completed, and outputting the bit matrix; constructing a bit matrix transpose function TransInv (.), an input and output bit matrix and an output bit transpose matrix; and dividing the output bit matrix into bit square matrixes, and performing bit granularity transposition on the bit square matrixes to obtain ciphertexts corresponding to the 512 groups of messages after SM4 encryption. According to the invention, the tower domain optimization technology is used to optimize the operation efficiency of the S box, and the bit slicing technology and the SIMD technology are combined to realize the parallel encryption and decryption of 512 groups of plaintext messages.

The invention relates to a method for rapidly extracting same name point information of unmanned aerial vehicle images in a coastal zone. The method comprises automatically determining a camera placement mode according to image data, estimating an image ground coverage range according to the global elevation grid data, carrying out coarse extraction on the image overlapping information, extractingconnection strength information between the images, networking the measurement zone images according to the impact matching results and producing and outputting the measurement zone same name point.The method can analyze image overlapping information according to the unmanned aerial vehicle image characteristics in the coastal zone, efficiently provides the same name point information for the subsequent aerial triangulation, utilizes a processing algorithm with a high parallelism degree, has a good data processing capability, utilizes a multi-threaded parallel technology in production of image thumbnails and grayscale images and image feature point extraction and matching and has high calculation efficiency.

The invention discloses a community discovery algorithm based on an improved association rule, and the algorithm comprises the steps: firstly carrying out the self-adaption of a support degree, and calculating the minimum support degree through a mathematic method; secondly, introducing a Boolean matrix and a transaction weight thought to improve an Apriori algorithm, and reducing the database scanning frequency; and finally, combining with a Spark platform to realize association rule improved community discovery algorithm parallelization. According to the community discovery algorithm based on the improved association rule, the community members are mined by using the MAC address. The Apriori algorithm is improved by introducing the idea of support degree self-adaption and adding a transaction weight to generate a Boolean matrix, the improved algorithm is combined with Spark to realize parallelization of the algorithm, and the relationship between community members is mined by mininga frequent item set. Experimental results show that the ARCD algorithm solves the problems of subjectivity of manual setting of support degree and redundancy of community mining results, has good expandability, and improves the mining speed of community discovery.

The invention discloses a multi-source data processing and fusion method and system for a low-voltage distribution network, and the method comprises the steps: firstly transforming a conventional neural network, proposing a Hermite orthogonal basis forward neural network according to the polynomial interpolation and approximation theory, building an algorithm model based on the Hermite orthogonalbasis forward neural network on the basis, and carrying out multi-source data processing and fusion of the low-voltage distribution network; performing algorithm parallelization under a MapReduce framework so that the real-time processing requirement of the low-voltage power distribution network on mass data can be better met, finally, verifying the model by simulation. The result proves that themodel is higher in low-voltage power distribution network multi-source data processing efficiency and more accurate in result.

The invention belongs to the technical field of parallel particle simulation methods, in particular to a method for eliminating data competition in parallel operations of particle simulation algorithms. The present invention solves by setting: the data competition influence range of a grid in the one-dimensional particle simulation algorithm Ns=2×nMax+1 grid, and nMax is the maximum value of the data competition influence range of a grid in the one-dimensional particle simulation algorithm ; Then add a new numbering for subsequent application of the method of removing data competition for all grids, the numbering rule takes the grid set with a length of Ns number as a unit, numbering 1-Ns in turn, and looping like this until all grids are traversed Then extract the grids with the same number and put them into a collection to generate Ns collections, each of the Ns collections has no data competition in all the grids; finally, the Ns collections are serialized implement. The invention significantly improves the parallel efficiency of the particle simulation algorithm by eliminating data competition.

The invention belongs to the field of algorithm parallelization, and in particular relates to an improved Canopy parallel algorithm implementation structure. The improved Canopy parallel algorithm implementation structure disclosed by the invention comprises the following steps that: a chain node connection structure is adopted; when data shows that a node is a strong node in a node belonging cluster, the node belongs to the cluster; next data is processed; or else, the data is transmitted to a next node, so that the next node is started; the above works are repeated till all the data is processed; the chain structure is formed from serial nodes for equally distributing works and forms an annular ring; each node continuously scans information sent by previous nodes; if being registered in a data buffer area, data waits for being processed; if the data in the data buffer area is processed and the data is still not transmitted after a certain time, the node is dormant automatically till the next node is awakened; strong clustering points of various clusters are stored below respective nodes; and weak clustering point sets and central points of all clusters are globally visible. By means of the improved Canopy parallel algorithm implementation structure disclosed by the invention, the communication traffic and the power consumption can be greatly reduced; and the operation speed is increased.