111 results about "Data reorganization" patented technology

The invention relates to the technical field of data processing, and discloses an ecological degradation restoration evaluation system and method based on multi-source data fusion, and the system comprises a heterogeneous data recombination module, a space-time reference adaptation module, an anti-noise dimension fusion module, a degradation feature coupling module, a restoration scheme mapping module and a restoration feedback evaluation module. Performing dynamic heterogeneous recombination on the target area remote sensing image, ground monitoring and historical ecological parameters to form an original data matrix; obtaining a space-time reference adaptive matrix through adaptive space-time reference conversion; a multi-source anti-noise data cube is formed through anti-interference dimension fusion; performing degradation characteristic multi-source coupling on the cube to generate a degradation characteristic topological set; performing spatial feature association mapping on the topology set and a repair case library to output a repair scheme optimization sequence; based on the historical restoration feedback flow, the sequence is dynamically fed back and recombined to generate an ecological restoration evaluation report; according to the invention, the accuracy of reference information generation can be improved.

The invention relates to the field of semiconductors, in particular to an answer generation method and device based on a semiconductor knowledge base and a medium. The method comprises the steps of performing data association on structured knowledge according to a semiconductor domain knowledge structure to construct a class of knowledge bases; performing data recombination on the unstructured knowledge according to the layout so as to construct a second-class knowledge base; a query question is received, retrieval is executed in the first-class knowledge base and the second-class knowledge base based on a preset retrieval model, and a plurality of related knowledge fragments are obtained; determining a first fusion weight between the knowledge fragments of the same knowledge type based on the relevancy between each knowledge fragment and the query question and the update time of each knowledge fragment; determining an optimal knowledge type according to the group feature information, and determining a second fusion weight between different knowledge types according to the optimal knowledge type; and based on the first fusion weight and the second fusion weight, obtaining a reference answer according to the plurality of knowledge fragments. And the requirements of the semiconductor field on information comprehensiveness and answer accuracy are considered.

The embodiment of the invention provides an order book data construction method and system based on an FPGA, and the order book data construction method based on the FPGA is applied to a programmable logic component, and comprises the steps: obtaining transaction quotation data, and decoding the transaction quotation data; performing data recombination on the decoded transaction market data according to a preset format to obtain market information; and synchronously sending the market information to a target memory and a target cache, and constructing order book data in the target cache based on the market information.

The invention discloses a method and equipment for compressing multi-dimensional data based on column storage and self-adaption, and the method comprises the steps: recombining original multi-dimensional data into a column data set structure, and organizing data block storage for the column data set structure according to a hierarchical structure; each data block stores observation grid data of a single variable of multi-dimensional data at a certain moment so as to support column type storage and self-adaptive compression; and carrying out real-time statistical analysis on characteristics or parameters of variables in the data blocks, and dynamically selecting and configuring a filter and parameters thereof to compress the multi-dimensional data. The data of the same variable are stored together through column storage, so that the data access efficiency is greatly improved, especially during variable-level analysis. Meanwhile, the self-adaptive compression technology can dynamically select a compression mode according to the distribution characteristics of the data, so that the compression rate of the data is effectively improved, and the occupied storage space is reduced. The processing speed of the meteorological grid point data can be obviously improved, and the storage cost is reduced.

The invention relates to the technical field of data processing, and provides a cloud data archiving method and system, and the method comprises the steps: obtaining a source data set of to-be-archived data, carrying out the grade division, obtaining a data grade label, dividing the source data set into a plurality of data blocks, carrying out the data encryption of different grades, obtaining a plurality of encrypted data blocks, and storing the encrypted data blocks in a database; analyzing and coding to obtain a multi-dimensional data storage structure, uploading the multi-dimensional data storage structure to a preset cloud storage node, and generating a storage index information set based on multiple data blocks in the uploading process; and when a data recovery request is received, acquiring the data block from the cloud storage node according to the storage index information set, and performing decryption and data recombination. Through the scheme, classification, hierarchical encryption and structured archiving of the source data set are realized, the security and management efficiency of the data in a cloud storage process are improved, and the problem that the encryption level cannot be dynamically adjusted by data sensitivity or content features when a large-scale heterogeneous data source is used is solved.

The invention discloses a data processing method and device, electronic equipment and a computer storage medium, and relates to the technical field of data acquisition, the method comprises the following steps: acquiring an acquisition data packet sent by each data acquisition module, and analyzing according to the acquisition data packet to obtain successfully analyzed data, the successfully analyzed data at least comprising an acquisition node identifier; determining a memory address identifier of the successfully analyzed data according to the acquisition node identifier and a preset Hash mapping table, and after the successfully analyzed data is stored in a memory buffer area mapped by the memory address identifier, performing data quality verification according to the successfully analyzed data to obtain a quality verification result; after the data quality verification is completed, data reorganization is carried out according to the successfully analyzed data to obtain reorganized data, data restoration is carried out on the reorganized data according to the quality verification result to obtain final output data, and the method and device aim at improving the processing capacity of large-scale data collection so as to ensure the accuracy and timeliness of data collection.

The invention relates to the field of cloud computing big data secure storage, in particular to a cloud computing-based big data secure storage method and system. The method comprises the following steps: performing encryption and fragmentation processing on an original data stream to generate a fragmentation sequence with an index; generating a dynamic key sequence and a verification key group based on the fragment information; constructing a secure transmission channel through a quantum key distribution mechanism in combination with a geographic position weight and a load balancing strategy, and realizing encapsulation and transmission of a secure fragment packet; node state monitoring and redundancy management are implemented, and an update mapping table is generated; and finally, a complete data stream is recovered through fragment positioning retrieval, data recombination and dynamic decryption. According to the invention, the security and reliability of data storage and the overall load balancing capability of the system are improved.

The invention provides a memory access system oriented to vector and matrix calculation, the system comprises a vector memory access unit, a matrix memory access unit, a vector register group and a matrix register group, the vector memory access unit is connected with a memory interface and the vector register group, reads elements of a one-dimensional data structure or a two-dimensional data structure from a memory, and stores the elements of the one-dimensional data structure or the two-dimensional data structure; vector data are generated through data reorganization operation, a matrix access unit is connected with a memory interface and a matrix register set, matrix block data of a two-dimensional data structure are read from a memory together with the matrix access unit, matrix data are generated after data reorganization, and the matrix data are broadcasted to one or more computing units according to rows or columns. And performing calculation on the matrix data and the vector data. In the memory access system, the vector memory access unit and the matrix memory access unit can load data in parallel, and the utilization rate is improved through a plurality of computing units, so that the problems of low memory access efficiency and low memory bandwidth utilization rate are solved.

The invention relates to the technical field of automatic scheduling, in particular to a method for realizing automatic gas scheduling based on a gas prediction model. The method comprises the following steps: acquiring furnace area sensing data, and performing data reorganization processing to obtain furnace area structured data; analyzing the temperature change amplitude and the pressure distribution condition of each furnace area according to the structured data of the furnace areas; identifying the furnace temperature jump point of each furnace area by using the temperature change amplitude of each furnace area; the working moment proportion of the furnace temperature jump point of each furnace area is counted, and a high-proportion furnace body is identified; according to the temperature change amplitude and the pressure distribution condition of the high-proportion furnace body, a furnace body gas flow velocity vector field is recovered, an instantaneous pressure loss sudden change path is calibrated, and a gas main network physical anomaly path is obtained; calculating a gas scheduling capability state according to the pressure distribution condition of the furnace area to obtain an actual scheduling structure of the gas main network; and screening a feasible gas distribution scheme for the physical abnormal path of the gas main network. According to the invention, the operation efficiency and adaptability of the gas main network can be improved.

The invention discloses a transformer substation work ticket knowledge graph construction method based on a large language model, and belongs to the field of energy power and artificial intelligence. The method comprises the following steps: based on standardized electrical drawing annotation data, generating an independent knowledge graph through data recombination preprocessing, rough knowledge graph construction and attribute supplementary reasoning; all the independent knowledge maps are divided into a framework layer map and a detail layer map according to voltage grades, and full map fusion is completed by adopting a hierarchical fusion strategy; carrying out breakpoint detection by using an isolated point detection algorithm based on degree centrality, carrying out connectivity analysis through depth-first search, and re-fusing multiple non-connected sub-graphs; and screening abnormal nodes based on an electrical connection rule, inputting the abnormal nodes into a large language model to automatically supplement Agents to generate correction suggestions, and completing map optimization. Through cooperation of the large model Agent and multiple algorithms, the electrical data structuring efficiency and accuracy are improved, and knowledge network support is provided for automatic operation and fault diagnosis of the transformer substation.

The invention discloses a multi-channel data transmission control method and system. The method comprises the following steps: acquiring current to-be-transmitted data and an idle channel set; judging the relationship between the number of channels and the data volume, and when the channels are sufficient and the data volume is large, splitting the data into reference data and corresponding split data by adopting an intelligent data division strategy; establishing a matching relationship between the data set and the channel; inputting the reference data and the split data into corresponding channels for transmission in a cross arrangement mode; and performing data fusion based on the same data field at a receiving end. Through dynamic channel matching, intelligent data splitting and optimization recombination mechanisms, the channel utilization rate and the transmission efficiency are effectively improved, the data recombination complexity is reduced, the method is particularly suitable for high-concurrency and low-delay data transmission scenes, and compared with a traditional method, the transmission performance can be remarkably improved.

The invention relates to a big data intelligent analysis processing system and method based on a distributed architecture, and the system comprises a data collection module, a hybrid processing module, an intelligent data distribution module, a data recombination module and a data analysis module which are connected in sequence. The method comprises the steps that real-time big data and batch big data are obtained from a multi-source heterogeneous data source through an increment extraction channel and a batch extraction channel respectively, the real-time big data are submitted into a message queue in two stages through the CDC technology, and the batch big data are stored in a distributed library in an increment mode according to timestamps / self-increment IDs; during hybrid processing, the intelligent resource scheduler makes a decision by means of a resource allocation function and cluster state evaluation, and the task dispatcher decomposes tasks and dynamically distributes the tasks in a weighted manner; high-frequency data and low-frequency data are divided according to rules and information entropies through intelligent shunting; according to the method, the time mapping weight fusion is established through data recombination, and finally the recombination data is analyzed, so that efficient resource scheduling, accurate data distribution and deep analysis can be realized, the real-time performance and the reliability are considered, and the data processing efficiency and the analysis precision are improved.

The present invention discloses a method, device, electronic device, and computer-readable storage medium for processing inertial navigation communication data. The method comprises: receiving packet data, intercepting the minimum and maximum packet sequence numbers, and all packet sequence numbers and corresponding status bytes between the minimum and maximum packet sequence numbers, and filtering out incoherent packet sequence numbers; sequentially treating the numbers 0-N as a data block, with each L number representing N being the number of valid bytes in the packet sequence number and L being the length of the complete parsed byte; and for each data item in the incoherent packet sequence number array, determining which data block the data item is located in, determining and deleting the packet sequence number subarray and status byte subarray corresponding to the data block, replacing the deleted packet sequence number subarray with the data block, and replacing the deleted status byte subarray with L zeros in the status byte array. The present invention can achieve dynamic compensation for packet loss and data reassembly, improving system performance and reliability in complex communication scenarios.

The application discloses a file unshelling method and electronic equipment, and relates to the technical field of computer security, and comprises the following steps: inserting a probe at a target system call of a kernel, and injecting the probe into a target program; identifying a suspiciously shelled process through the target program; monitoring a memory permission change behavior performed by the suspiciously shelled process through the target program, and counting a permission change frequency of a memory region, an entropy value of the memory region and a call chain depth; when it is confirmed that the suspiciously shelled process is in an unshelling stage according to the permission change frequency, the entropy value and the call chain depth, writing data decrypted by the suspiciously shelled process into a target buffer through the target program; reading the data from the target buffer in a user mode, and recombining the data into a memory image according to a base address and a dirty page bitmap and writing the memory image back to a file. The method can avoid I / O bottlenecks caused by full memory dumping, reduce invalid data in the user mode, and reduce CPU occupation and memory bandwidth.

The present application provides a kind of data distribution device, data distribution system and data distribution program.Data distribution device (2) has: update data acquisition unit (9a), obtains the multiple update data corresponding to multiple write object electronic control device;Reorganization information acquisition unit (9b), obtains the reorganization information for by main device parallel execution to multiple write object electronic control device transfer update data transfer processing;Data packet generation unit (9c), multiple update data are segmented based on reorganization information to generate segmented data, and the segmented data of multiple update data generated are packed to generate multiple data packets;And distribution control unit (9e), multiple data packets are distributed to main device.

The invention relates to the technical field of film and television media. The 8K video real-time editing cooperation method and system for the 10-gigabit hybrid network are provided, and the method comprises the steps: carrying out the incremental analysis processing of an editing behavior based on operation event monitoring, and generating an incremental data unit; performing versioning management processing on the incremental data units through distributed storage nodes, and constructing a storage system with a version chain; according to the recovery instruction, target data recombination processing is carried out based on the storage system with the version chain, and the engineering environment is reconstructed; and performing merging processing based on the conflict characteristics in the engineering environment to generate a repaired engineering version, thereby achieving the technical effects of reducing the network transmission load, improving the conflict resolution efficiency and enhancing the version management reliability.

The application discloses a kind of mask protection circuit, device, chip and electronic equipment, it is related to information security technical field, in the circuit: timing control module is used to generate timing control signal based on the input system clock signal, and timing control signal is transmitted to routing encoding module;Data buffer module is used to receive the first bit width external input data, and after converting external input data into the second bit width intermediate input data, intermediate input data is transmitted to routing encoding module;Routing encoding module is used to recombine intermediate input data, generate the third bit width target input data, and according to timing control signal, target input data is distributed to compression accumulation tree module;Compression accumulation tree module includes multiple compression accumulation tree units, each compression accumulation tree unit is used to operate target input data based on double-track logic, and outputs the fourth bit width share pair operation result.This application can effectively reduce circuit area and reduce calculation delay.

The invention provides an intrusion detection device and method, electronic equipment and a storage medium, and belongs to the technical field of network security, and the method comprises the steps that a data collection module obtains a data frame of a network interface through multiple buffer areas, and sends the data frame to a protocol analysis module; the multiple buffer areas are a plurality of data buffer areas working alternately; the protocol analysis module carries out data recombination processing on the data frame, generates a to-be-detected data packet, and sends the to-be-detected data packet to the anomaly detection module; the anomaly detection module performs anomaly feature matching on the to-be-detected data packet based on the executable detection object to generate an intrusion detection result; the executable detection object is obtained by analyzing the rule description file based on the rule loading module; according to the method and the device, data acquisition is performed by adopting a plurality of alternately working data buffer areas, parallelization and buffering of data read-write operation are realized, and a data recombination and rule pre-analysis technology is combined, so that the problem of packet loss when the network flow is relatively large can be effectively solved, and the security of the system is improved.

The invention discloses a radiotherapy information management system based on multiple data sources and a construction method thereof, and relates to the technical field of medical information, and the system comprises a multi-source data obtaining module which is used for obtaining discrete data of each heterogeneous system; the multi-source data intelligent aggregation engine is used for recombining discrete data into format data and developing a dynamic form generator; the decision support rule base is internally provided with verification rules and is used for carrying out real-time verification and abnormal intervention on the format data; and the three-screen linkage workbench is used for integrating visual linkage browsing of patient information, image data and dose planning. According to the invention, full-link automatic integration and multi-system collaborative optimization of patient data are realized, so that the accuracy, efficiency and expandability of a radiotherapy process are improved.

The application discloses a substation work ticket knowledge graph construction method based on a large language model, and belongs to the field of energy power and artificial intelligence. The method comprises the following steps: based on standardized electrical drawing annotation data, independent knowledge graphs are generated through data reorganization preprocessing, rough knowledge graph construction and attribute supplement reasoning; all independent knowledge graphs are divided into framework layer graphs and detail layer graphs according to voltage levels, and hierarchical fusion strategies are adopted to complete full graph fusion; breakpoint detection is carried out by using an isolated point detection algorithm based on degree centrality, connectivity analysis is carried out through depth-first search, and non-connected subgraphs are re-fused; abnormal nodes are screened based on electrical connection rules, a large language model is input to automatically fill in Agent to generate correction suggestions, and graph optimization is completed. Through the cooperation of the large model Agent and multiple algorithms, the application improves the efficiency and accuracy of electrical data structuring, and provides knowledge network support for substation automation operation and fault diagnosis.

This invention relates to the field of communications, and in particular to a method, apparatus, and system for encrypted transmission of game data. The method achieves semantic changes in data packets by swapping characters across data packets in a data packet group. Thus, even if the sub-data is intercepted by a thief, the data packet reassembled by the thief according to the number is a scrambled data packet with changed semantics, which misleads the thief and prevents them from knowing the true data content, thereby ensuring data security.

The invention provides a data processing method of a column storage database and a related product. The data processing method comprises the following steps: performing a hash strategy-based set operation on a column storage database to obtain an intermediate result; under the condition that the capacity of the intermediate result exceeds a preset memory threshold value, creating a temporary file based on a line storage structure in a disk; variable-length data columns are obtained from the intermediate result column by column, each variable-length data column comprises multiple rows of variable-length data, and each row of variable-length data comprises a data length value, a data numerical value and a flag bit; performing data recombination on each row of variable-length data in the obtained variable-length data column so as to reduce the data length of the variable-length data; and writing the variable-length data column after data recombination into a temporary file. According to the data processing method, the data length of the variable-length data is reduced, the space occupation of the variable-length data column of the intermediate result of the set operation based on the Hash strategy during disk falling is reduced, and the disk falling performance of the system is improved.

The application discloses a kind of three-dimensional ground penetrating radar array antenna array element time delay synchronism evaluation and correction method, comprising the following steps: step 1, full reflection scene simulation in air medium:step 2, data acquisition: three-dimensional ground penetrating radar is opened, saves the data of all array elements of array antenna not less than 30 seconds;Step 3, data reorganization and interception;Step 4, array element time delay synchronism evaluation:step 5, each array element time delay synchronization correction coefficient calculation.The method disclosed in the application preliminarily establishes three-dimensional ground penetrating radar array antenna performance evaluation system, is the complement of general ground penetrating radar performance evaluation, makes up the existing evaluation method cannot accurately evaluate three-dimensional ground penetrating radar array antenna each array element synchronization degree, array element time delay difference size and whether it will affect the quality and display effect of subsequent various three-dimensional stereoscopic view.

The application discloses a DMA accessing method and device supporting online data reorganization, and relates to the technical field of data processing, which comprises the following steps: when a DMA engine receives a DMA transmission instruction, it is judged whether there is a data reorganization instruction in the DMA transmission instruction; if yes, the data transmission direction and data reorganization parameter information are obtained; the DMA engine analyzes and splits the received DMA instruction into an access request and sends the access request to an upstream storage in the transmission direction; when the DMA engine receives access response data sent by the upstream storage, the response data is reorganized and stored in a data buffer in the DMA engine; when the data buffer collects all data in the current DMA transmission instruction, the data is forwarded to a downstream storage in a reorganized format according to the data reorganization parameters in the DMA transmission instruction. The application supports online data reorganization, makes the data reorganization process transparent to the operation core, provides convenience and flexibility for programmers, and reduces the implementation cost and power consumption of data format conversion.

The application provides a Redis high-hot data migration method, which comprises the following steps: counting original request information sent from a host to one or more slaves corresponding to the host in a Redis cluster, asynchronously analyzing the original request information and pushing the analyzed request information; recombining the pushed analyzed request information to count the access pressure of each data in the one or more slaves; and obtaining the load condition of the host, and when it is determined that the load of the host meets the predetermined standard of the host, migrating the data with the top N access pressures in all data in all slaves to a backup slave, wherein N is a positive integer. The application can realize automatic migration of high-hot data in a Redis cluster, and avoid the slave crash and cluster crash caused by all access quantities hitting the same slave in the prior art.

An iCloud data decryption and extraction system and method, relating to the field of data security technology, includes: verifying user compliance authorization and initializing the decryption environment; obtaining the user's root key based on a temporary access token and the user's iCloud account compliance authorization credentials; deriving session keys based on the user's root key and iCloud data storage structure information to obtain temporary session keys for various data types; performing parallel multi-threaded scheduling on the data to be extracted; matching decryption algorithms for various types of data downloaded to the decryption end and performing data decryption operations based on the temporary session keys; performing data download integrity verification and data decryption integrity verification; when both verifications pass, performing data reconstruction operations and reconstructed integrity verification; and outputting and securely destroying the extracted complete data types, significantly improving data extraction efficiency and data decryption compatibility and integrity.

This invention provides a training method for a distribution network voltage prediction model and a method for predicting distribution network voltage. The method includes: acquiring a training set; training a preset machine learning model based on the training set to obtain a first machine learning model; when the quality of the sample data in the training set does not meet a first preset requirement or the first prediction result of the first machine learning model does not meet a second preset requirement, performing data correction and data recombination processing on the training set based on the probability density function of the training set to obtain a recombined training set; using the recombined training set as a new training set, re-executing the steps of obtaining the first machine learning model based on the training set and subsequent steps until the quality of the sample data meets the first preset requirement and the first prediction result meets the second preset requirement, and then using the first machine learning model as the target distribution network voltage prediction model. This invention can train a high-precision distribution network voltage prediction model, thereby achieving accurate voltage prediction at each node in the distribution network.

The invention discloses an industrial data endogenous security method, system and equipment based on a physical unclonable function and a dynamic entropy pool. The method comprises the following steps: generating an unclonable device root key at an industrial device end by using a PUF technology; constructing a dynamic entropy pool fusing the environment, the data and the service entropy; deriving a one-time session key based on the device root key and the dynamic entropy value; encrypting the industrial data by adopting a post quantum cryptography and symmetric cryptography hybrid mechanism; and carrying out dynamic slicing and confusion processing on the ciphertext to generate a data packet containing a mixed slice and an encryption mapping table. And at the access end, authority verification is performed through the block chain smart contract, a temporary access token is issued, and data recombination and decryption are completed by using the token. According to the invention, internal generation of the security slave device is realized, a full-link, self-adaptive and anti-quantum computing security guarantee from slave device identity and data generation to storage access is provided, and the method is especially suitable for a high-value and high-sensitivity industrial manufacturing environment.

The invention discloses an optimization method and system for improving the bandwidth of a neural network accelerator, and relates to the technical field of neural network accelerators, and the optimization method for improving the bandwidth of the neural network accelerator comprises the following steps: carrying out the convolution processing of input data through a convolution pipeline; performing activation function processing on the data subjected to convolution processing by adopting a single data processor; an on-chip buffer is adopted to replace a traditional on-chip memory for caching, and pooling processing and local response normalization processing are carried out on the data after activation function processing through a plane data processor and a channel data processor; and performing data format conversion on the data subjected to the local response normalization processing through a data recombination engine. According to the invention, by optimizing the storage read-write process of the data when the accelerator performs acceleration operation, the utilization rate of memory bandwidth is effectively improved, the time consumed by single acceleration operation is reduced, and the performance of the accelerator is improved.

The application provides a kind of data security efficient access method and system, comprising: receiving the access request of user, receiving the access request of user;With multistage recursive aggregation search algorithm, according to access request, in turn through global index, cross-device index and dynamic local index data search is carried out, and the summary search data is obtained;With the improved block encoding homomorphic encryption method combined with data reorganization rules, the search data is fed back to the user;The application is based on global index, cross-device index and dynamic local index, improved block encoding homomorphic encryption method combined with data reorganization rules, composed of the micro-service architecture of power grid, through micro-service architecture, based on access request, carries out layer by layer search, realizes the safe and efficient access of power equipment data, effectively solves the efficient retrieval of data across regions and across devices in distributed power system, solves the problem of safe transmission of data.