18 results about "Merge sort" patented technology

The invention relates to the technical field of electrical digital data processing, and discloses a statistical entropy and multi-factor fused text similarity data processing method, which comprises the following steps that: a processor extracts substring sets which do not contain maximum common values of a first data sequence and a second data sequence, and calculates the quadratic sum of the lengths of substrings to generate local statistical entropy; traversing the maximum common substring set to obtain storage address indexes of the maximum common substring set in the first data sequence memory space and the second data sequence memory space, and constructing a topological mapping vector of a mapping structure displacement relationship; calculating the total number of inverted pairs of the topology mapping vector by using a merge sorting algorithm, and generating a normalized topology dissipation index; and by taking the local statistical entropy as an information carrier and taking the topological dissipation index as a structural damping factor, executing nonlinear damping modulation operation to obtain a final similarity score, and solving the technical problem that the block-level displacement cannot be identified by linear scanning logic by quantizing topological entropy increase of data distributed in a storage space.

The invention discloses a production line data integration method based on a digital twin model, and belongs to the technical field of electronic data processing. The production line data integration method comprises the following steps: step S0, pre-preparation and standard definition; the method comprises the following steps: S1, data acquisition and preprocessing; step S2, data cleaning and integration; s3, data analysis and mining; 4, constructing a digital twinborn model; and 5, managing system-level data. The method has the following advantages: data islands are cracked, data quality and processing efficiency are improved, internal business logic of data is mined, high-precision feature extraction and pattern recognition are realized, and the method is suitable for popularization and application by means of unifying data standards and cleaning rules, optimizing a merge sorting strategy, building a double-layer association analysis model, adopting a CNN-LSTM hybrid model, perfecting data security and authority management and the like. The method supports the cross-line collaborative decision and precise production optimization, finally solves the problem that the prior art cannot support the system-level digital twinning application, and improves the application efficiency and effect of the digital twinning system.

The invention discloses a data processing device, a data processing method, a chip and electronic equipment, and relates to the technical field of data processing. The data processing device comprises a storage module which is configured to store x groups of original data vectors, the original data vectors comprise m data elements, and x and m are positive integers; the calculation module is configured to access the original data vectors stored in the storage module, and reconstruct the x groups of original data vectors into one or more intermediate matrixes according to the parallel processing capability of the single-instruction multi-data execution unit and a target value k of a Top-k operator; loading the intermediate matrix into a vector register block according to columns, carrying out odd-even merging sorting by utilizing a single-instruction multi-data execution unit, and realizing in-line sorting on the intermediate matrix; and loading the intermediate matrix subjected to in-line sorting into a vector register block according to columns, and performing one or more rounds of merging sorting by utilizing a single-instruction multi-data execution unit to obtain top-k data elements in x groups of original data vectors.

The invention discloses a multi-energy and carbon emission correlation analysis method and system for a high-energy-consumption enterprise, and relates to the technical field of energy management and carbon emission accounting, and the method comprises the steps: carrying out the preprocessing of historical multi-source data of a high-energy-consumption target enterprise, and constructing a training set and a test set which are distributed in a balanced manner; then introducing a shuffled frog-leaping optimization algorithm to perform global optimization on the network structure and parameters of the deep extreme learning machine, determining the optimal hidden layer structure and weight configuration of the model through an iteration mechanism of subgroup division, local jump and merge sorting, and constructing a multi-energy consumption prediction model with efficient deep feature extraction capability; a coupling feature matrix generated based on real-time multi-source data is input into a multi-energy consumption prediction model to obtain a multi-energy consumption prediction result, finally, association analysis is performed by combining real-time data, the prediction result and an energy carbon emission coefficient, the association strength of multi-energy and carbon emission is quantified, and the energy consumption prediction accuracy is improved. And more accurate and reliable data support is provided for energy structure optimization and emission reduction decision making of high-energy-consumption enterprises.

This invention relates to the field of artificial intelligence technology, providing a data sorting method, electronic device, storage medium, and computer program product. The method includes: performing local sorting of data to be processed in parallel using multiple processing units of a parallel processor; each processing unit acquiring a portion of the data to be processed; sorting the acquired portion of data and selecting a preset number of data points as local target data; and performing one or more merge sort operations on the local target data acquired by each processing unit, based on the amount of data to be processed and the hardware resources of the parallel processor, and selecting a preset number of data points as global target data from the merged and sorted data. This invention avoids the huge computational overhead of globally sorting all data to be processed, significantly improving the efficiency and performance of TopK operations.

This application relates to a hardware-implemented sorting method, apparatus, computer device, readable storage medium, and program product. The method includes: acquiring a first data sequence to be sorted, where each data element in the first data sequence contains a key value and its position index within the sequence; concatenating the key value and position index of each data element according to a preset sorting direction to obtain the first data elements to be sorted; and performing hardware parallel sorting on the first data elements using a bitonic merge sort network to obtain a sorted second data sequence, wherein the sorting order of the data elements in the second data sequence retains the relative positional relationships in the first data sequence. This allows the primary sorting order to be determined based on the key value, and the relative order of data with the same key value to be determined based on the positional information. While maintaining sorting stability, it inherits the hardware parallel advantages of bitonic merge sort, resulting in fast sorting speed suitable for real-time video decoding requirements.

The application discloses a distributed database incremental snapshot method and device and computer equipment. The method comprises the following steps: obtaining a change log from a distributed database node, wherein the change log comprises transaction key information; assembling the change log to obtain an assembly result; using a transaction start timestamp as a sorting key, performing merge sorting on the assembly result through a log structure merge tree structure to generate an ordered lake format file; using a Paimon interface to generate a snapshot ID reflecting a current database state based on a transaction start timestamp or a table time field; and querying the ordered lake format file and the snapshot ID by using a standard data lake reader. The method of the application can not only simplify the data migration process from the distributed database to the data lake, but also improve the efficiency and reliability of the whole process, and ensure the consistency and integrity of the data transmission between different systems.

The embodiment of the invention discloses a switching control method, a chip and an energy storage system, and the method comprises the steps: sorting state parameter groups of all energy storage modules in the energy storage system, and obtaining a plurality of first state parameter groups after in-group sorting; performing at least one level of merging sorting on each first state parameter group after sorting in the group to obtain each state parameter after merging sorting; determining a to-be-switched-out energy storage module and a to-be-input energy storage module in the current control period from the energy storage modules based on the merged and sorted state parameters; and based on the to-be-switched-out energy storage module and the to-be-input energy storage module, performing switching control on each energy storage module.

The application relates to a data sorting method and device, a computer device, a storage medium and a computer program product. The method comprises the following steps: obtaining an initial data sequence, dividing the initial data sequence to obtain a plurality of basic sub-data sequences; taking each basic sub-data sequence as an initial sub-data sequence, combining each initial sub-data sequence to obtain a plurality of initial sub-data sequence combinations; performing merge sorting on each initial sub-data sequence in the same initial sub-data sequence combination to obtain an intermediate sub-data sequence combination corresponding to each initial sub-data sequence combination; taking the intermediate sub-data sequence combination as an initial sub-data sequence, returning to the step of combining each initial sub-data sequence until a termination condition is met, and obtaining an ordered data sequence corresponding to the initial data sequence. The method can improve the data sorting efficiency and can be applied to a multidimensional database.

The invention discloses a dialogue content generation method and electronic equipment, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a target vector which is output by an activation function of a dialogue model created based on a pre-training model and is constructed based on the sampling probability of each lexical element in a word list, carrying out the partitioning of the target vector, and obtaining a plurality of data blocks, and then, based on a target sampling probability threshold, screening out the first lexical units in each data block in parallel to construct corresponding target sets, then, merging and sorting the target sets to obtain a target list, and generating and outputting corresponding target dialogue content based on the target list. According to the method, the target vector output by the activation function is partitioned, then the lexical units in the data blocks are screened out in parallel to construct the corresponding target set, and then the target set is merged and sorted to obtain the target list to generate the corresponding target dialogue content, so that the hardware utilization rate is effectively improved, the data granularity of single processing is reduced, and the dialogue generation efficiency is improved.

The application provides a dynamic table merging method and device, equipment and storage medium, relates to the technical field of data processing, and the merging method comprises the following steps: obtaining an original list and a merging condition, and determining label data and non-label data in the original list according to the merging condition; wherein the original list comprises at least two original data groups, the original data group comprises a plurality of original data, the original data in the same column corresponds to the same data node; according to the label data and the non-label data, determining to-be-merged data corresponding to the label data and non-merged data corresponding to the non-label data; performing merging sorting on the to-be-merged data corresponding to the label data of the same type to generate merged data; and generating a target list according to the label data, the non-label data, the merged data, the non-merged data and the original list. The application performs merging processing on complex list data, reduces the calculation difficulty of the number of cross cells, and greatly reduces the merging calculation difficulty.

The invention relates to the technical field of database query and distributed systems, and discloses a performance optimization method and system for cross-data-source paging query, and the method comprises the steps: receiving a query request, and retrieving a historical deviation calibration factor; sending a statistical probe to the heterogeneous data source, aggregating the statistical probe into a global distribution estimation function, and calculating a physical value domain anchor point; constructing a differential locking window, and distributing a prefix counting instruction and a window data acquisition instruction in parallel; calculating a global reference displacement according to a returned counting result, merging and sorting window records, and intercepting a target result set; and calculating the error between the real ranking and the estimated ranking of the physical anchor points, and updating the historical deviation calibration factor according to the error. According to the method, logic offset is mapped into physical anchor points, index counting and local slicing are used for replacing linear scanning, and a deviation feedback self-learning mechanism is combined, so that IO overhead of cross-source deep paging query is remarkably reduced, and continuous optimization of query performance and positioning precision is realized.

The application discloses a dialogue content generation method and an electronic device, and relates to the technical field of artificial intelligence, and comprises the following steps: obtaining a target vector constructed based on sampling probabilities of each word element in a word table and output by an activation function of a dialogue model created based on a pre-training model; obtaining a plurality of data blocks by blocking the target vector; filtering out first word elements in each data block based on a target sampling probability threshold to construct a corresponding target set in parallel; performing merge sorting on each target set to obtain a target list; and generating and outputting corresponding target dialogue content based on the target list. By blocking the target vector output by the activation function, filtering out the word elements in each data block to construct the corresponding target set in parallel, and then performing merge sorting to obtain the target list to generate the corresponding target dialogue content, the hardware utilization rate is effectively improved, the data granularity of single processing is reduced, and the dialogue generation efficiency is improved.

The application discloses a sequencing duplicate read marking method and system based on block-level division and external sorting, and the method comprises the following steps: logically dividing a BAM / CRAM file, and allocating a continuous and data-equivalent file block to each thread; each thread constructs a ReadEnds data structure containing read position information, sequence numbers and sequencing quality sums in parallel, and outputs the ReadEnds data structure to a temporary file after being sorted and compressed; performing external multi-path merge sorting on the temporary file to generate a globally ordered data stream, and performing parallel scanning to identify duplicate sets, mark duplicate reads and extract sequence numbers; performing merging processing on the boundary duplicate sets, and performing external sorting to obtain a sequence number data stream; and each thread writes back the original file according to the sequence number data stream in parallel, sets a mark on the duplicate reads and outputs a result file in the original sequence. The application realizes full-process parallel processing, controllable memory and high I / O efficiency, and can stably and efficiently process super-large-scale sequencing data.

The invention is suitable for the technical field of databases, and relates to a data deduplication method and device, computer equipment and a computer readable medium. In the import stage, generating a current data fragment corresponding to the imported data record, and distributing a logic sequence number for the current data fragment according to a generation sequence; according to the primary key of the data record in the current data fragment, generating a sorting key for merging and sorting, and removing the repeated data record in the current data fragment in the merging process; in the submitting stage, for each current data fragment, other data fragments with small logic serial numbers are selected in sequence, and according to the sorting key corresponding to the current data fragment and the sorting keys corresponding to the selected other data fragments, repeated data records in the selected other data fragments are recognized and removed until all other data fragments are traversed. According to the method and the device, data de-duplication can be realized under the condition that a primary key index does not need to be maintained, so that the memory occupation is reduced, and the processing efficiency of the system is improved.

The application relates to the technical field of electric digital data processing, and discloses a text similarity data processing method fusing statistical entropy and multiple factors, which comprises the following steps: a processor extracts a maximum common sub-string set not containing each other of first and second data sequences, calculates a local statistical entropy by calculating the square sum of sub-string lengths; the maximum common sub-string set is traversed to obtain storage address indexes thereof in the first and second data sequences, and a topological mapping vector of mapping structure displacement relationship is constructed; the total number of reverse order pairs of the topological mapping vector is calculated by using a merge sorting algorithm to generate a normalized topological dissipation index; the local statistical entropy is taken as an information carrier, the topological dissipation index is taken as a structure damping factor, and a nonlinear damping modulation operation is performed to obtain a final similarity score; and the application solves the technical problem that a block-level displacement cannot be recognized by linear scanning logic by increasing the topological entropy of the distribution of quantitative data in a storage space.

The application discloses a data processing device, a data processing method, a chip and an electronic equipment, and relates to the technical field of data processing. The data processing device comprises a storage module configured to store x groups of original data vectors, wherein each original data vector comprises m data elements, x and m are positive integers; and a calculation module configured to: access the original data vectors stored by the storage module, reconstruct the x groups of original data vectors into one or more intermediate matrices according to the parallel processing capability of a single-instruction multiple-data execution unit and a target value k of a Top-k operator; load the intermediate matrices into a vector register group column by column, perform odd-even merge sorting on the intermediate matrices by using the single-instruction multiple-data execution unit, and realize in-row sorting of the intermediate matrices; load the intermediate matrices after in-row sorting into the vector register group column by column, perform one or more rounds of merge sorting on the intermediate matrices by using the single-instruction multiple-data execution unit, and obtain top-k data elements in the x groups of original data vectors.

The invention discloses a quantum anonymous multi-party ranking method based on a quantum merge ranking algorithm and application thereof, and belongs to the technical field of quantum security multi-party computing. The method aims at solving the technical problems that an existing quantum anonymous multi-party ranking scheme depends on a semi-honest third party, quantum resource consumption is large, transportability is poor, and the anti-attack ability is insufficient. According to the method, a non-collusion node constraint + modular quantum operation framework is constructed, two non-collusion participants are used for guaranteeing safety, and privacy protection coding is achieved in combination with d-dimensional single particle state, quantum Fourier transform and permutation / shift operation. According to the method, under the condition that no third party participates, distributed data anonymous ranking is dynamically completed on the premise that data privacy and identity association are not leaked by a plurality of participants. Meanwhile, the method is compatible with mainstream quantum sorting algorithms such as quantum merge sorting through modular design, quantum resource consumption is only bound with the size of a data set (independent of the upper limit of a data value), and the resource efficiency is remarkably improved when facing a large-range data set.