340results about How to "Reduce data redundancy" patented technology

The present invention discloses parallel mass data transmitting middleware of Internet of things and a working method thereof. The invention includes: a data collection module, for forwarding data collected from different types of sensors to an application interaction module after analysis; the application interaction module, for achieving communication between the parallel mass data transmitting middleware of the Internet of things and an application system of the Internet of things, analyzing a control command of the application system of the Internet of things, transmitting to the data collection module, and further transmitting the analyzed sensor data to a data management module; the data management module, for performing data screening, fusion, conversion on the mass data analyzed by the application interaction module, and finally storing the calculated data into the application system of the Internet of things; and a developing regulation module, for setting out a protocol plug-in regulation, a communication channel plug-in regulation, a service field metadata plug-in regulation, etc. The present invention can meet high concurrency access and management of the large scale sensing apparatus of different application fields, various communication channels and various communication protocols.

The invention discloses a traffic-road condition information processing method based on a vehicle wireless sensor network. The method is characterized in that: firstly, a wireless communication network is established and the network includes an information center, a vehicle wireless sensor network sub-system formed by limited vehicle mobile knots and an information publishing sub-system based on a broadcasting network; the information center is fixed in a traffic hub in a city as the center of the wireless communication network; secondly, each vehicle mobile knot collects the timely road condition information and integrates data through the vehicle wireless sensor network sub-system according to the instruction of the information center; then the timely road condition information from the information collection and data integration of each road section is reported to the information center through the vehicle wireless sensor network sub-system; finally, the processed traffic road condition information and network control instructions are published by the information center through the information issuing sub-system to each vehicle mobile knot for the road choosing by drivers.

The invention discloses a file processing method, which comprises the following steps of: encoding a file into a plurality of data blocks by utilizing an erasure code algorithm, and storing the plurality of data blocks in at least two storage nodes which at least comprise storage nodes in a distributed file system; wherein one data block is only stored once; and storing the file index of the file,the respective storage address of each data block in the plurality of data blocks and the hash value related to the file in a block chain network. Therefore, the data redundancy is greatly reduced, higher data reliability is obtained with lower data redundancy, and the utilization rate of the storage space is improved. Moreover, the method adopts a mode of combined storage of the block chain network and the out-of-chain distributed storage system, so that the storage pressure of the block chain network is reduced, and the storage capacity is expanded by introducing the distributed file system. The invention further discloses a file processing device, equipment, a medium and a computer program product.

The invention is suitable for the field of storage and provides a file uploading method and device, and a mass storage system. The method comprises the following steps: a terminal device generates summary information corresponding to the file to be uploaded by using a preset summary information generating algorithm according to the file to be uploaded, and uploads the summary information to a storage server; the storage server searches whether the summary information uploaded by the terminal device exists or not; and when the storage server does not find the summary information, the terminal device uploads the file to be uploaded to the storage server. In the embodiment of the invention, the terminal device needs to upload the file to the storage server only when the file to be uploaded does not exist in the storage server, thereby reducing the data redundancy in the storage server, saving the network resources while uploading files and increasing the file uploading speed.

The invention discloses a two-stage single-instance data de-duplication backup method. De-duplication of data at two stages is performed during backup. The method comprises firstly, performing repeated data detection on a file scale, inquiring local logs to judge whether identical files are stored or not, and if the identical files are stored, informing users to complete the backup operation; if identical files are not stored locally, informing backup programs of server ends to inquire databases to judge whether files with identical content exist or not, if the files with identical content are searched, establishing links pointing to the files for clients only, and recording quotes of the files by clients by the server ends; if the files are new, uploading the files and recording information of the files by two ends; further processing the files after the files are uploaded to the server ends by background programs, and splicing small files together to avoid waste of space; storing large files respectively by type, comparing similar files regularly, and performing difference de-duplication at the second stage after grouping.

The invention discloses a method and a system for intrusion detection. Cluster analysis is carried out on a test sample set through a graphical partition method, then the behavior type of cluster obtained through the cluster analysis on the test sample set is determined by utilizing a training sample set, and thus the determination of the behavior type of the test sample set is realized. Since the determination process of the behavior type of the test sample set does not completely depend on the training sample, contained in a training sample set, owned behavior type, a new behavior type can be found by adopting the scheme provided by the embodiment of the invention.

The invention provides a component quality checking device based on line laser three-dimensional measurement and detection method of the device. The quality checking device includes a mobile platform mounted on a guide rail and capable of sliding along the guide rail at a uniform speed. To-be-checked components are fixed on the mobile platform. A camera and a plurality of laser emitters are mounted above the guide rail through a mount. The camera collects images which are casted to the surfaces of the to-be-checked components by the laser emitters and deformed due to surface modulation. The camera transmits the collected image data to a computer for analysis and treatment. According to the detection method, by casting linear stripes emitted by the laser emitters to the surface of the to-be-checked components for linear scanning and by utilizing the camera for collecting image information on the to-be-checked components continuously, measured three-dimensional point cloud data and initially designed three-dimensional image are compared and analyzed in the same coordinate system for judging whether the maximum error is within the maximum permissible error range or not and screening out qualified components. The device provided by the invention is high in detection precision and realizes automatic and flow line production.

Vector quantization provides font contour data reduction. Characters are defined in a font collection using segmented outline forms and common references stored to curve segments for similar segments in different characters of the font collection. Compressing the font collection preferably includes applying an entropy encoding data reduction. The common references are stored in a segment storage location. Another aspect includes decompressing a font store containing a compressed collection of outline fonts. The uncompressed curve segments are scaled so as to match a requested size for a letterform.

The invention provides a data inquiry method and system, consensus method and system, apparatus and storage medium. Each node of a block chain network respectively stores a Merkel tree of each block,and a global index table corresponding to the Merkel tree of each block. The version number of the version is a block height, wherein the query instruction includes the queried first block height; a target data is look up in that global index table according to the height of the first block. By configuring the corresponding Merkel tree, The global index table with the block height as the version number, and searching the target data in the global index table through the queried block height, can complete the query of the sub-node data of Merkel tree leaf in the block chain through one read operation, thus greatly improving the data reading performance of the system.

The invention discloses a traffic safety management (TSM) information multihop broadcasting method. The method comprises the following steps of: detecting a traffic safety event vehicle to serve as a source node to broadcast TSM information; calculating a reference waiting time slot Tn of a highway section in which a node is positioned by the node which receives the TSM information according to a weight factor, namely the distance between the node and the source node, dynamically regulating additional forwarding waiting time Delta n according to an estimated value of link quality signal to interference plus noise ratio (SINR), and calculating and acquiring waiting time tn of the node for transmitting a message; and when the waiting time tn is reduced to zero, immediately broadcasting request traffic forward (RTF), after a previous forwarding node receives the RTF, determining a single relay node to transmit a data packet, and immediately forwarding the received TSM information after the node receives the RTF. By the method, link quality information is combined, data forwarding reliability is fully guaranteed, the phenomenon of forwarding collision between multiple nodes in the same highway section is reduced at the same time, and the link load and data redundancy in a media access control (MAC) layer are effectively reduced based on an RTF/authorized traffic forward (ATF) forwarding confirmation mechanism.

In a system and method for facilitating wireless communication between agents in a wireless network, each agent is equipped with an agent's device (10a, 10b) using a main wireless infrastructure (2) and a separate auxiliary wireless infrastructure (1). Initial contact between any first agent's device (10a) and any other agent's device (10b) is made within the auxiliary wireless infrastructure, whereas, once this initial contact has been established, all ensuing communications between the first agent's device (10a) and the other agent's device (10b) take place within the main wireless infrastructure.

The invention provides a data storage method and a system, an apparatus and a storage medium. The method comprises the following steps: a global index table corresponding to a Merkel tree of each block is stored locally at each node of a block chain network; and a Merkel tree of each block is stored locally. Where the version number of the version is the block height, and the global index table isused to query data according to the block height. By configuring the corresponding global index table for the Merkel tree and taking the block height as the version number, the invention provides a data inquiry way for the system to inquire the global index table directly through the block height and complete the inquiry through one reading operation, thereby greatly improving the data reading performance of the system. And further optimizes the robustness of the system in the event of transaction congestion or node crash.

The invention discloses a light stream optimization based three-dimensional reconstruction method and device. The method comprises the following steps of: collecting a depth picture and a color picture of a multi-viewing angle by utilizing a multi-viewing-angle depth camera, and carrying out three-dimensional spatial alternation so as to obtain a point cloud of each viewing angle; calculating measuring vectors of two groups of point clouds of adjacent viewing angles according to the information of the depth picture and the color picture; establishing a light stream optimization formula according to a multi-channel light stream method by utilizing the measuring vectors of two groups of point clouds of adjacent viewing angles; resolving the light stream optimization formula, thereby obtaining the matching result of two groups of point clouds of the adjacent viewing angles; filling each viewing angle point cloud into a fusion matrix according to the matching result; and processing the fusion matrix, thereby obtaining the reconstruction result. According to the invention, as less as possible of three-dimensional structures of point characterization models can be utilized self-adaptively, thus data redundancy is lowered; meanwhile, the model accuracy is high; and additionally, hardware requirement is not high, and the application range is wide.

The invention provides a multi-cluster cooperative data processing method and system, a memory medium and a computer device. The method comprises the steps of receiving collection data and convertingthe collection data into structured data; establishing a basic data sharing area on a Hadoop cluster and loading the structured data to the basic data sharing area; and cooperatively processing the data in the basic data sharing area through utilization of the Hadoop cluster, a Hadoop online processing cluster and a plurality of distributed database clusters according to a data application scene,thereby providing corresponding data service and finishing a data processing process. According to the method, the system, the memory medium and the computer device, the data redundancy can be reducedthrough establishment of the basic data sharing area.

The invention relates to a system and a method for trajectory tracking during circulation of an E-mail. The method includes: when a sender sends an E-mail, a unique identifier is generated for each receiver, and the receiver accesses the E-mail theme through the unique identifier; when each receiver's operation on the E-mail is detected, the operation trajectory is fed back to the sender; after the information feedback, the sender can accurately learn about whether the corresponding receiver reads the E-mail or not, whether the corresponding receiver forward the E-mail or not, whether the corresponding receiver download an attachment or not, whether the corresponding receiver print the attachment or not, whether the corresponding receiver edit the attachment or not, E-mail reading times, E-mail reading time, E-mail forwarding times, E-mail forwarding time, who the E-mail is forwarded to, attachment reading times, attachment reading time, attachment downloading times, attachment downloading time, attachment editing times, attachment editing time, attachment printing times, attachment printing time and the like. By the method, comprehensive E-mail trajectory tracking can be performed, and E-mail transmitting efficiency and safety are increased by the information feedback.

The invention discloses a high-performance remote sensing image synthesis method. The method comprises the steps that original remote sensing image data is acquired, and a cloud covered area in the original remote sensing image data is extracted; mask clipping is performed on the remote sensing image data obtained after extraction, and cloudless fragment image data is obtained and stored; a task area to be synthesized is divided into blocks to obtain multiple subtask areas; the cloudless fragment image data in the subtask areas is subjected to substitutive inlaying parallelization processing according to cloudless inlaying rules, and synthesized cloudless image data blocks in all the subtask areas are output; and the cloudless image data blocks acquired from all the subtask areas are gathered to output a synthesized cloudless remote sensing image of the whole task area. According to the method, only a cloudless fragment image is stored, so that the data storage volume is lowered; substitutive inlaying processing is performed according to areas in a multithread parallel mode, thus, efficiency is improved, and a large-range cloudless inlaid image can be acquired; by means of local histogram matching, the problem of hue inconsistency is eliminated while hue adjustment is guaranteed.