29 results about "Aggregation methods" patented technology

The invention belongs to the technical field of power grid dispatching, and particularly discloses a dual-port equivalent cluster aggregation method. According to the method, an independent aggregation operation interval is established for two ports by solving a specific robust optimization problem, an initial aggressive boundary is formed, and an initial high-dimensional polyhedral model is established. The key improvement is that the condition extreme value of the power of the other port is solved under the constraint of the operation interval of one port, so that the physical coupling relation is converted into a series of accurate boundary points, and meanwhile, the calculation complexity is also reduced. And finally, cutting the initial and aggressive high-dimensional model by using the boundaries, eliminating an infeasible space generated by neglecting coupling, and quickly generating a feasible dual-port aggregation interval. The problems of large model error and low aggregation efficiency caused by dual-port interconnection are systematically solved.

The invention discloses an intelligent aggregation method and engine for multi-source data, and relates to the technical field of data aggregation, and the method comprises the steps: firstly carrying out the segmentation and vectorization mapping of a multi-source data stream containing resumes, news and documents, retaining an original index, and analyzing a fine-grained atomic fact through a fine-tuning large model; then, cross-source alignment and clustering are carried out on the fact tuples, deep logic implication reasoning and conflict detection are executed on the basis, and the support and exclusion states among facts of different sources are quantified; furthermore, truth value optimization is completed by using a global consistency scoring mechanism, and a high-confidence core fact is locked from complex contradictory information. And finally, generating an abstract based on a preferred fact, and establishing an evidence traceability link from the generated content to the source data by using the original index. Thus, logic verification can be fused into the generation process, conflicts and model illusion of multi-source heterogeneous data in the space-time dimension are effectively eliminated, and logic self-consistent, true, credible and traceable intelligent aggregation is achieved.

The invention relates to the technical field of the Internet of Things, and discloses a method, a system and equipment for aggregating multi-dimensional data with dynamic dimension expansion in the Internet of Things, and a medium. According to the method, through an elastic modulus set and a pre-distributed redundant prime number, historical data and global parameters do not need to be reconstructed when dimensions are newly added, and optimized Paillier homomorphic encryption and CRT coding are fused, the calculation burden of the encryption process can be effectively reduced, and through batch verification at edge nodes, the communication and calculation overhead is remarkably reduced, and the communication efficiency is improved. A multi-layer data verification mechanism is provided by integrating Bulletprofs zero knowledge proof and a threshold Schnorr signature, false data injection, man-in-the-middle attack and data tampering can be effectively prevented, the integrity and authenticity of data in the aggregation process are ensured, the confidentiality of original data is ensured by using homomorphic encryption and CRT coding, and even if an aggregation ciphertext is intercepted, the security of the aggregation ciphertext is ensured. And an attacker cannot decrypt a single data value.

According to the load aggregation method and device based on the network load graph, by constructing the dynamically updated network load graph and establishing the network load coupling relation model, the constraint of a power grid topological structure on load adjustment is accurately quantified, and matching of a regulation and control strategy and a real-time system state is ensured; by introducing a multi-dimensional comprehensive aggregation evaluation system, the differences of the load in the aspects of characteristics, economy, response speed and the like are fully considered, and refined portraying and accurate aggregation of load resources are realized; and finally, through combination of multi-time scale aggregation and a multi-objective optimization algorithm, system operation indexes which conflict with each other are effectively coordinated, and an optimal load aggregation scheme which can quickly respond to system changes and can comprehensively optimize the regulation and control effect, the economic cost and the response speed is generated.

Systems and methods for partitioning forecast items into segments, or groups, based on an attribute. Once partitioned, segments are aggregated based on a predetermined memory size limit using an aggregation method. Aggregation methods include methods for providing segments that have a similar number of records. Segmentation is further enhanced by including variance reduction gain as a metric for selecting the order of attributes.

Embodiments of the present application provide a data aggregation method, a data aggregation device, an electronic device, a chip, a storage medium and a computer program product. The method comprises: performing a desensitization aggregation operation on target data in a target database according to aggregation query configuration information to obtain a desensitized aggregation data table; wherein the aggregation query configuration information comprises: a dimension field, a metric field, a desensitization rule and an aggregation rule; the dimension field and the metric field represent the target data, the desensitization rule represents a desensitization manner of a specific field, and the aggregation rule represents an aggregation manner of a specific field.

This invention relates to the field of database management technology, providing a streaming aggregation method, system, and device in a database. The method includes: establishing a mapping relationship in the database; responding to a user creating a streaming view containing aggregation operations, determining the corresponding final calculation function based on the mapping relationship, and creating a materialized table for storing intermediate aggregation states; rewriting the streaming view as a query that executes the final calculation function on the materialized table; responding to data being written to the streaming table, generating a first query plan based on the original query of the streaming view, generating a second query plan and a third query plan based on the mapping relationship, and executing them sequentially to first obtain the intermediate states of the current write operation, then merging the current write intermediate states with the existing intermediate states in the materialized table and updating the materialized table; responding to a user querying the streaming view, executing the rewritten streaming view, obtaining and returning the aggregation result. This invention can fully implement streaming aggregation functionality without modifying the database kernel.

The application provides a power hierarchical routing aggregation method and system, which defines an aggregation window, configures a total timeout time, a tentative correlation parameter for the aggregation window, and initializes a loss count, a tentative probability and a waiting set of each sub-flow, starts the aggregation window to receive sub-flow data, and makes tentative judgment on the aggregation window according to the receiving result, sends a pre-warning packet to a missing sub-flow through the tentative result and the tentative probability, collects a pre-warning reply, updates the loss count, the tentative probability and the waiting set of the sub-flow through the pre-warning reply, waits for the sub-flow according to the update state of the waiting set, updates the loss count and the tentative probability of the sub-flow in the waiting set through the waiting result, and performs data packet aggregation forwarding after the waiting set is empty and the tentative initiation condition is not met or after the aggregation window times out, resets the aggregation window and performs the next round of aggregation after forwarding, so that the balance among aggregation efficiency, data integrity and resource saving is achieved through the power hierarchical routing aggregation method.

The application provides a traffic flow prediction method combining spatial multi-scale and time sequence multi-granularity structure, and belongs to the field of space-time data mining. First, the original data is preprocessed, and a suitable space-time aggregation method is selected to obtain multi-space scale data and multi-granularity time sequence data. Second, the multi-space scale data under the time sequence granularity of the original data is input into a multi-space feature mining module to learn and fuse the multi-scale space correlation between traffic units. Then, the multi-time sequence granularity data under the multi-space scale is input into a multi-time feature mining module to learn and fuse multi-granularity traffic time information. Finally, a multi-time and space feature fusion module is used to effectively aggregate the multi-space scale and multi-time traffic mode features, and output the final prediction result. Compared with the traditional method in the embodiment, the application has obvious advantages in prediction accuracy, can effectively improve the traffic flow prediction efficiency, and can realize efficient prediction.

The invention discloses a tensor clustering and aggregation method for dynamic equivalent modeling of a wind power plant. The method comprises the following steps: firstly, establishing a detailed model of the wind power plant and simulating a dynamic trajectory of the detailed model; selecting dynamic trajectory data of each wind driven generator state in the wind power plant to construct a tensor data set for describing the working conditions of the wind driven generators; an improved CP decomposition algorithm is adopted to carry out dimensionality reduction on the tensor data set, wherein the dimensionality reduction retains original structure information; clustering the wind driven generators on the dimension-reduced data set by using a k-means clustering algorithm; the wind driven generators belonging to the same category are equivalent to one aggregated wind driven generator; and calculating equivalent current collection system parameters to obtain a wind power plant equivalent model comprising the aggregated wind power generator and the equivalent current collection system. The tensor method is used for fan clustering, the high-dimensional characteristic of the dynamic trajectory of the wind power plant is considered, and the accuracy of the aggregation model is effectively improved.

The invention provides an NAS-based data verification, restoration and aggregation method and system, and the method comprises the steps: generating a real-time data flow according to the real-time access data and real-time shared data of each user in a network in the NAS, and carrying out the preliminary verification to obtain a plurality of vulnerability character strings of the real-time data flow, then deducing access behaviors / sharing behaviors of the vulnerability character strings, generating related behavior characteristics corresponding to each access behavior / sharing behavior according to the real-time data stream, performing character repair on the corresponding vulnerability character strings, and performing data segment division and classified storage; according to the method, the migration guide corresponding to the data source is constructed according to the migration track corresponding to each data segment, and the storage node corresponding to the suspicious data segment is adjusted according to the migration difference between the migration guides corresponding to each data source at different moments, so that the data security, the repair reliability and the access efficiency of the NAS system are improved.

The invention provides a federated learning robust aggregation method and system for resisting backdoor attacks, and belongs to the technical field of network and information security. The method comprises the following steps: acquiring model update vectors submitted by a plurality of clients, and decomposing the model update vectors into a plurality of hierarchical update data according to the hierarchical relationship of a global model; according to each hierarchy update data of each client, extracting a multi-dimensional feature of the client; determining a global total abnormal score of the client according to the multi-dimensional features of the client; and calculating a comprehensive reputation of the client according to the global total abnormal score, determining an aggregation weight according to the comprehensive reputation, aggregating updating data of each level of the client according to the aggregation weight, and updating the global model. According to the method, the attack success rate of malicious updating can be remarkably reduced, and the method has the advantages of better robustness and safety.

The application relates to a wind farm dynamic equivalent aggregation method based on PCA and Kernel-FCM and belongs to the technical field of power system control. Specifically, the method comprises the following steps: obtaining the unit dynamic characteristics in a preset wind turbine mathematical model, selecting multiple clustering indexes according to the unit dynamic characteristics, calculating the initial values of state variables corresponding to the clustering indexes, constructing high-dimensional data samples from the initial values of the state variables, performing dimension reduction processing on the high-dimensional data samples based on a PCA analysis method to obtain low-dimensional data samples, determining the optimal clustering number by balancing simulation accuracy and calculation amount based on a Gap Value, performing cluster division on the low-dimensional data samples based on a Kernel-FCM algorithm, calculating dynamic equivalent parameters of a power collection network based on a fast analytical method, and constructing a wind farm dynamic equivalent aggregation model based on the equivalent unit parameters and the dynamic equivalent parameters of the power collection network. The application can improve the reliability and accuracy of clustering results.

The application provides a multi-constraint-based spatial element dynamic aggregation method and system, and belongs to the technical field of spatial data processing. The method comprises the following steps: acquiring shared bicycle parking point spatial element data set and multi-dimensional constraint conditions; performing spatial profiling on a target research area based on spatial division constraints, generating a plurality of initial aggregation service range units, and mapping elements in the spatial element data set into the initial aggregation service range units, and calculating statistical indexes of the initial aggregation service range units; constructing a graph structure with the initial aggregation service range units as nodes, and the spatial adjacency relationship and semantic similarity between the units as edges; constructing a graph structure aggregation model, including a target function with the minimum variance of the statistical indexes between the initial aggregation service range units as the target, and a constraint condition based on the integrity of the spatial division constraints; and solving the graph structure aggregation model by using a heuristic search algorithm or a graph partitioning algorithm to generate a dynamic aggregation service range result satisfying the statistical balance constraint.

This invention relates to the field of e-commerce data processing technology, specifically to a method and apparatus for multi-platform e-commerce order aggregation based on dynamic clustering rules. The technical solution includes: collecting order data from e-commerce platforms and performing standardized processing to obtain platform order data; extracting various clustering feature values ​​from a clustering feature system; matching these values ​​with rules in a dynamic clustering rule set to determine the candidate cluster to which the order belongs; calculating the comprehensive similarity between orders; determining whether the comprehensive similarity is greater than or equal to a preset similarity threshold, and assigning the order to that cluster; performing missed order detection for orders not clustered; and outputting the order results for the clusters. This significantly improves clustering accuracy, effectively reduces missed orders, and minimizes operating and transportation costs. It also significantly improves order processing efficiency, adapting to the needs of large-scale order processing. The system operates stably and supports efficient processing of massive amounts of data.

The invention provides a federated learning robust aggregation method and system fusing layer-by-layer dynamic clustering and statistic adaptive preferential selection, and belongs to the technical field of federated learning. The method can dynamically cluster and screen benign update layer by layer: after a server receives client model update, the benign update is accurately identified through a hierarchical clustering mechanism, a randomized clustering algorithm selection strategy is introduced, each layer of clustering algorithm is dynamically determined, and clustering robustness is enhanced; based on a hierarchical clustering result, a statistical magnitude calculation mode (such as a mean value and a median) is selected through a random number, so that random statistical magnitude self-adaptive preferential selection is realized; the situation that a single statistic is dominated by an abnormal value is avoided, the influence of abnormal data on global model updating is reduced, multi-source information is fused to generate global model updating, abnormal value interference is effectively restrained, and the adaptability to complex data distribution is improved.

The invention belongs to the intelligent power grid industry, particularly relates to the technical field of intelligent scheduling systems, and discloses an equivalent cluster dual robust aggregation method considering source load uncertainty. The method comprises the following steps: firstly, establishing an equivalent cluster model containing uncertainty; the feasibility of the scheduling scheme under the worst uncertainty is ensured through the first min-max robust optimization; secondly, the worst influence of uncertainty on instruction execution is actively evaluated through second max-min robust optimization, the scheduling instruction tends to be conservative by maximizing the correction cost, and therefore the aggregation interval is actively reduced; and finally, combining the results of the two reconstruction modes to carry out iterative solution until convergence. According to the method, the finally generated aggregation operation interval is more compact and reliable while the uncertainty influence is fully considered, it is ensured that any scheduling instruction in the interval can be safely executed when facing any uncertainty, and the problem that the interval reliability is poor is fundamentally solved.

This invention provides a data verification, repair, and aggregation method and system based on NAS, comprising: generating a real-time data stream in the NAS based on each user's real-time access data and real-time shared data in the network, performing preliminary verification to obtain several vulnerability strings in the real-time data stream, then deriving the access / sharing behavior of the vulnerability strings, generating relevant behavioral characteristics corresponding to each access / sharing behavior based on the real-time data stream, performing character repair on the corresponding vulnerability strings and dividing and classifying the data segments for storage, constructing migration guidelines for the corresponding data source based on the migration trajectory corresponding to each data segment, and adjusting the storage nodes of the corresponding suspicious data segments based on the migration differences between the migration guidelines corresponding to each data source at different times, thereby improving the data security, repair reliability, and access efficiency of the NAS system.

The invention discloses a multi-dimensional data screening and aggregation method based on user question intention, and relates to the technical field of data processing, the method comprises the following steps: firstly receiving user commerce and trade questions and performing semantic analysis, and extracting a core operation intention and an intention feature set; determining a multi-dimensional data demand and generating a commerce and trade exclusive data demand dimension set according to an inherent corresponding relationship between commerce and trade operation behaviors and commercial elements, and determining an association relationship between dimensions; then, through basic data screening and integration and data consistency verification, a commerce and trade exclusive initial data set and a verification data set are generated in sequence; performing multi-dimensional correlation operation on the verification data set, and removing redundant data to obtain a commerce and trade exclusive target aggregation data set; and finally, according to the commerce and trade decision presentation logic conversion data, generating a commerce and trade exclusive structured answer and feeding back the answer to the user side.

The application discloses a database parallel aggregation method based on a column storage format, a computer device and a storage medium, and comprises the following steps: dividing to-be-aggregated data into a plurality of data blocks; assigning all the data blocks to a plurality of grouping tasks to perform grouping operations on row positions in parallel, to obtain a plurality of sub-tables; for any one of the grouping tasks, grouping row position information according to a first column aggregation field value to obtain a first sub-table including a plurality of records, for each record of the first sub-table, obtaining row position information in a row position sequence from the record, and obtaining a corresponding aggregation field value from a second column aggregation field value according to the obtained row position information, and re-grouping row position information according to the aggregation field value on the basis of the first sub-table to obtain a second sub-table including a plurality of records; and the like; merging the sub-tables of all the grouping tasks to obtain a plurality of shards; and merging all the shards to obtain an aggregation result.

The application discloses a federated learning aggregation method based on MD sampling and model credibility, an MD sampling method is used for sampling edge devices, sampling is performed based on the ratio of the sample quantity of the edge devices to the total sample quantity, a server sends a model to the selected edge devices, the sampled edge devices perform local training, local model updating is performed through a stochastic gradient descent method, and relevant parameters of the local training are uploaded after a local training number is reached.A global model aggregation strategy based on model credibility is as follows: firstly, the model credibility of the uploaded local model of the edge devices is calculated; the model accuracy of the local model is calculated based on the local model accuracy rate; the historical accuracy model of the local model of each edge device is calculated based on a time attenuation function; and finally, the model credibility and the local historical model accuracy are used as the proportion of the local model in aggregating the global model.The application guarantees the accuracy and safety of the model and improves the convergence rate of the model.

This disclosure provides an accumulation method, system, and storage medium. The method includes: allocating a pair of buffers, with a first pointer and a second pointer pointing to the pair of buffers respectively; for each first row in a first sparse matrix containing a plurality of non-zero elements, identifying a plurality of second rows in a second sparse matrix corresponding to the plurality of non-zero elements; obtaining a plurality of intermediate lists calculated based on each non-zero element and the corresponding second row; storing the plurality of intermediate lists in the first buffer; performing an iterative process, including: merging the plurality of intermediate lists in the first buffer into a smaller number of intermediate lists; storing the smaller number of intermediate lists in the second buffer; swapping the first pointer and the second pointer; determining whether an exit condition for exiting the iterative process is met; and migrating a final merged list as a row of the output matrix of a sparse matrix-matrix multiplication from the first buffer to system memory. This disclosure reduces memory complexity and memory overhead.

The application discloses an anti-malicious fast security aggregation method and device, and relates to an anti-malicious fast security aggregation method, which comprises the following steps of S1, each of a plurality of participants obtains a plurality of random numbers, wherein the number of the participants is the same as the number of the random numbers obtained by each participant; S2, each participant uses the plurality of random numbers to generate a corresponding number of pseudo-random numbers for a current aggregation round, and calculates the generated pseudo-random numbers to obtain pseudo-random number calculation results; S3, each participant processes to-be-aggregated data according to the pseudo-random number calculation results, and all the participants send the processing results to a server for aggregation calculation; and S4, each participant receives the aggregation calculation results returned by the server, processes the aggregation calculation results, and obtains the aggregated results.

The invention provides a hierarchical federated learning-oriented cache aggregation method and apparatus, and a storage medium, the method is executed by an edge node in hierarchical federated learning, and the edge node is in communication connection with a terminal device and a global aggregation server; for each global training round, the method comprises the following steps: receiving local model data uploaded by a terminal device corresponding to the edge node; under the condition that the quantity of the local model data cached in the cache window corresponding to the edge node is smaller than the window size of the current round, adding the received local model data into the cache window; and under the condition that an aggregation condition is met, aggregating the local model data cached in the cache window to obtain aggregated model data, and uploading the aggregated model data to a global aggregation server to obtain global model data. According to the invention, on the premise of ensuring the model training precision, the defects of high communication overhead and low bandwidth utilization rate of the existing hierarchical federation learning can be overcome.

The invention discloses a multi-source data aggregation method and system for realizing hierarchical query, and relates to the technical field of data processing. The method comprises the steps of obtaining multi-source data of each storage unit of a target enterprise, and performing characterization according to the multi-source data to obtain multi-source features; substituting the multi-source features into an intelligent function model to obtain feature function scores; performing data important hierarchy division on the multi-source features through feature function scores to obtain a plurality of hierarchies; carrying out aggregation optimization on the multi-source features of any same level of each storage unit to obtain the aggregation credibility of each storage unit; performing abnormal feature filtering on the multi-source feature according to the aggregation credibility of each storage unit to obtain a first multi-source feature; performing weighted aggregation calculation according to the aggregation credibility to obtain an aggregation weight, and performing weighted averaging calculation according to the aggregation weight and the first multi-source feature to obtain an aggregation feature; generating an aggregation result according to the aggregation characteristics; according to the invention, the aggregation efficiency and accuracy of the multi-source data are improved.

The invention relates to the technical field of big data, can be applied to the technical field of medical health and finance, and discloses a log sampling dynamic aggregation method and device, computer equipment and a storage medium. The method comprises the following steps: when a service of an application node changes, obtaining log information collected by a log collector deployed on the application node; performing multi-dimensional importance scoring on the log information according to abnormal semantic features, business criticality, influence range and transmissibility and index mutation relevance to obtain a scoring result; and uploading the log information with the scoring result meeting the requirement and the corresponding scoring result to a log server. By implementing the method provided by the invention, a multi-dimensional strategy can be fused, a context sensing capability and a dynamic response mechanism are provided, the resource use efficiency can be improved, the fault response speed can be accelerated, the traceability of logs can be enhanced, and the method can better serve a complex and sensitive business environment.

The invention provides a model aggregation method and device, equipment, a medium and a product, and relates to the technical field of big data. The method comprises the steps that data of at least one first data party and data of at least one second data party are subjected to joint generalization, then split according to data provided by all the parties and returned to clients of all the parties, and after the clients of all the data parties conduct anonymization on the data of all the data parties, privacy intersection is conducted to obtain intersection data. And after training the initial model according to the local data corresponding to the intersection data in the federated learning task by each data party, transmitting an encryption gradient between clients corresponding to each data party to obtain a final model. Data generalization is enhanced while data availability is guaranteed through joint generalization, data leakage can be avoided by splitting and returning after joint generalization, encryption gradient is adopted, decryption into plaintext is not needed, accuracy and integrity of model updating are guaranteed, and safety and reliability of the federated learning model are improved.

This invention discloses a data-mechanism hybrid-driven method for modeling and aggregating feasible regions of demand-side resources. First, a hybrid model of "base power-state dynamics" for demand-side resources is established. Then, using historical operational data, the parameters of the hybrid model are precisely calibrated through parameter identification to form a personalized dynamic feasible region. Next, the personalized feasible region is mapped to a standardized descriptor. Finally, for a scenario tree describing future uncertainties, a probability-weighted aggregated feasible region is calculated using a data-driven aggregation mapping function. This invention can accurately quantify and efficiently aggregate the flexibility of distributed resources, and has promising engineering applications.

The invention discloses an importance sampling ratio aggregation method, electronic equipment, a medium and a product, and relates to the technical field of reinforcement learning, and the method comprises the steps: firstly, under the same target input condition, obtaining first probability information of a first sequence outputted by a current task model, and second probability information of a second sequence output by the historical task model. In addition, a currently used target aggregation model is obtained; then, after the first probability information and the second probability information are compared, behavior change information of the current task model compared with the historical task model is determined; under the condition that the target aggregation model is the first aggregation model, the target aggregation model can be directly adopted to aggregate the importance sampling ratio of the at least one lexical element position included in the behavior change information to obtain a target aggregation value so as to calculate a loss value. In this way, in the training process, weak points of the task model can be repaired, and the success rate of task execution of the task model is increased.