33 results about "Hash join" patented technology

The invention discloses a parallel query processing method for a share-nothing database cluster in a cloud computing environment. The method comprises the following steps of: firstly, segmenting a query plan to obtain sections which are executed by all data nodes; secondly, performing section processing on all data nodes to obtain Hash joins of all data nodes; and thirdly, performing Hash aggregation on control nodes by flowing of aggregated data between all data nodes to obtain a query result set. By the flowing operation of node data, the data can flow between all nodes in the query execution process, the parallel execution query process is finished, and an execution speed is greatly increased.

A hash integration system includes a hash join module including build and probe inputs. A hash aggregation module may aggregate on the probe input of the hash join module, and a hash table generation module may generate an integrated hash table including a record with values from the build and aggregated probe inputs. The hash join module may join the build and aggregated probe inputs to form a joined output.

The invention relates to a network training method for video data on the basis of deep learning and hash coding. The method comprises the following steps: extracting a feature matrix of a video sample by using a deep neural network; carrying out modeling by using the acquired feature matrix of the video sample as a whole body to obtain high-dimensional real value representation of the video sample; and further representing the obtained high-dimensional real value representation into binary hash coding by using a deep network.

The invention relates to a model training method based on deep learning and hash coding. The method comprises the steps of regarding partially labeled image data as training data of a network model, and representing the training data as an analog-two-value hash code through a deep network, wherein the analogy-two-value hash code refers to a simulation two-value hash code of which the value is a continuous value; regarding the obtained analog-two-value hash code as input, connecting the analog-two-value hash code to one or more task layers, and using one or more tasks to conduct training at the same time; obtaining a two-value hash code which is used for representing the training data and carries feature information capable of being retrieved based on the analog-two-value hash code.

The invention provides a Hash join method and device. The method comprises the steps of S1, acquiring first data to be operated in a first table to be operated and second data to be operated in a second table to be operated; S2, calculating the Hash value of each first datum, determining a partition number corresponding to each first datum according to the Hash value of each first datum, calculating the Hash value of each second datum, and determining a partition number corresponding to each second datum according to the Hash value of each second datum; S3, reading data with the same partition number into the same zone in a memory; S4, conducting Hash join on the first data and the second data in the same zone. By the adoption of the Hash join method and device, join efficiency can be improved.

The invention discloses a Hash join operator acceleration method and system based on an FPGA-DDR, and belongs to the field of acceleration of a memory database. The technical problem to be solved is how to realize acceleration operation in a construction stage and a detection stage of a Hash connection algorithm. The method comprises: storing a tuple, a hash table and a linked list through a DDR memory, hash calculation is conducted through an FPGA chip, the hash table and the linked list are updated, and the FPGA chip is controlled to conduct interactive communication facing the DDR memory; under the cooperation of the DDR memory and the FPGA chip, parallel multi-thread operation is executed in the construction stage and the detection stage of the hash connection to realize the construction and matching of the hash table. The system comprises a DDR memory and an FPGA chip, the DDR memory is used for storing a tuple, a hash table and a linked list, and the FPGA chip is used for storinga base address, executing hash calculation, updating the hash table and the linked list and controlling the DDR memory to perform interactive communication.

The invention relates to a face image retrieval method and device based on deep learning and Hash coding. For the problems of insufficient facial feature expression and insufficient feature discrimination in existing facial image retrieval, a facial space network and facial space loss are provided to automatically mine a facial region with discrimination in a facial image, so that the influence ofbackground information is reduced. Meanwhile, the Hash network learns the internal relation between the face features and the Hash codes, the face images are mapped into the Hash codes, and the calculation and storage cost of retrieval is remarkably reduced. The multi-scale face feature channel enhancement module in the Hash network enhances the dimension of strong distinguishing power in the face features. According to the alternate training strategy provided by the invention, the two networks are organically fused together, the information loss between face feature extraction and hash codegeneration is reduced, the distinguishing power of the face features is enhanced, meanwhile, the distinguishing power of the generated hash codes is enhanced, and the accuracy of face retrieval is improved.

The invention relates to the technical field of the method for realizing infringement resource extraction and identification comparison by using technical ways to prevent infringement copying and diffusion of picture contents, especially to a method involved in copyright protection of the picture content. The technical method is implemented as follows: step one, carrying out picture resource crawling; step two, carrying out picture feature extraction by using deep learning and coding the picture by using hash coding; and step three, carrying out picture code comparison by using a two-level hash code positioning technology to identify whether the content is an infringement content.

The invention discloses a deep learning and Hash coding video retrieval method, and the method is characterized by comprising the following steps of: performing shot segmentation and feature extraction by using the same feature; in video retrieval, shot segmentation takes video frames of the same shot as redundant frames to be processed so as to reduce the calculated amount and improve the retrieval efficiency, and features obtained through a shot feature extraction algorithm are taken as the basis of shot segmentation so as to solve the problem that features of different frames in the same shot are different. The method has the advantages that the video retrieval at the lens level can be realized, and the retrieval accuracy can be improved; The video shot retrieval accuracy in the aspectof video retrieval capacity reaches 95% or above, pure binary operation is adopted, and the speed can be increased by four times.

The invention discloses an APUF security authentication method based on polynomial reconstruction. The method comprises two stages of registration and authentication. At the registration stage, a server randomly generates a large number of incentives and a random number k, and an APUF generates corresponding response to serve as an abscissa. After CRC encoding, a corresponding real point is foundfrom a polynomial using the k as the coefficient, and a large number of hash points are added to form a point set V. The incentives, the point set V and a polynomial coefficient hash (k) subjected tothe hash are stored in the server. At the authentication stage, a PUF circuit on the hand of a user generates response, randomizes the response, and queries the corresponding real point in the point set V for constructing a polynomial so as to obtain a polynomial coefficient (the formula is described in the specification). Contrast authentication is performed on the (the formula is described in the specification) and the hash (k), and the authentication is successful if the two coefficients are the same. According to the APUF security authentication method disclosed by the invention, by meansof the response to the randomization processing by the APUF and the adding of the hash points, the machine learning attack can be resisted to ensure the security and the reliability of the data.

The invention provides a data connection method across a data center of an HDFS (Hadoop Distributed File System). The method comprises the following steps that: a coordination node acquires the IP (Internet Protocol) addresses of metadata nodes from a configuration file after receiving a data connection operating request, establishes connection with the metadata nodes in the configuration file one by one, and acquires metadata information about a requested metadata node when a requested table is inquired in a current metadata node; the requested node filters data of the requested node according to information about requested data, screens the requested data of the requested node, and transmits the size of a result set saved by the requested node to the coordination node; the coordination node informs small result set nodes to transmit result sets saved by the small result set nodes to all large result set nodes; all the large result set nodes perform Hash join on the result sets of the large result set nodes and the result sets transmitted by all the small result set nodes to obtain result records; and the coordination node aggregates the result records. Through the method and a system, data connection across the data center can be realized.

The invention provides an electrocardiogram data display method and a terminal apparatus. The method comprises the steps of acquiring the waveform of each heart beat from electrocardiogram signals, performing dimension reduction processing on the waveforms, and mapping the waveforms in a low-dimension space into hash points corresponding to the waveforms in a one-to-one manner according to preset constraint conditions; displaying a scatter plot including the hash points. Compared with the conventional Lorenz plot which is drawn via R-R intervals and can only reflect the exception of rhythms, the scatter plot formed in the embodiments of the invention enables waveform exceptions to be discovered and enables hash points to be displayed more clearly in limited space.

A method for detecting and handling skew and spillover in in-memory hash join operations. To improve the detection of skew and spillover in parallel processing systems, a Poisson distribution of unique hash values to Units of Parallelism (UoPs) is employed to determine the number of rows per UoP and in turn, the potential of spillover at a UoP. Hash join plan options can be selected or adjusted to reduce the likelihood of spillover.

The invention discloses a data association query method and system in a RocksDB-based key-value storage mode and belongs to the technical field of operator optimization. The technical problem to be solved by the invention is how to improve the efficiency of traditional hash join calculation based on line-by-line reading and serial operation. The associated query efficiency is improved, the adoptedtechnical scheme is as follows: the method comprises the following steps: ; the method is applied to a key-value storage system. Each row of data is stored through a channel of go language, and thenthe data is read one by one to realize block reading, i.e., the data is processed by taking a block as a unit, and after the data block is received, the data is distributed to different Join Workers for parallel computing, so that the computing performance of a Hash Join operator is improved; the method specifically comprises the following steps: S1, reading data block by block; s2, all the modules are in cache communication through channels, all the modules are made to be executed asynchronously, and all the modules are made to end execution by sending end marks; and S3, calculating the mutual concurrent execution of the Join Workers.

The invention belongs to the technical field of databases, and particularly relates to a method for pushing down predicate in SQL (Structured Query Language) connection operation, a terminal and a medium. The method comprises the following steps: carrying out Hash join on a fact table and a dimension table, generating a Hash table according to the dimension table, and generating a fact table histogram according to the fact table; according to the Hash table and the fact table histogram, obtaining N pieces of predicate histogram areas, wherein N is a positive integer; transforming the N piecesof predicate histogram areas into predicates, and pushing down the predicates to a data storage layer. Compared with minimum value/ maximum value and integral Hash push-down, the method can improve data filtering ability according to the fact table histogram so as to improve the calculation efficiency of the storage layer.

The invention discloses a parallel Hash connection acceleration method and system based on an FPGA, and the method comprises the steps: reading a plurality of tuple data from a first data table in each set period of a Hash connection construction stage, calculating a partition number corresponding to each tuple data through a pre-division function, guiding each tuple data to a corresponding partition, and reading one piece of tuple data from each partition to construct a hash table and a chain table; reading a plurality of tuple data from the second data table in each set period of a detection stage of hash connection, calculating a partition number corresponding to each tuple data by utilizing a pre-division function, guiding each tuple data into a corresponding partition, and reading a hash table and a chain table constructed in the tuple data detection matching construction stage from each partition, wherein the construction stage and the detection stage of hash connection are both completed in the FPGA. According to the method and the device, address conflicts do not exist when the tuples among the partitions access the hash table, and the capability of processing multiple tuples in parallel is realized.

In one aspect, a computer-implemented method includes detecting, by a server includes one or more processors, a request to perform a hash join operation on a data structure stored in a data storage device, forming a hash lookup dictionary based on lookup results in a hash table, storing the hash lookup dictionary in a cache, and probing, during a probing phase of the hash join operation, the cache.

The invention discloses an aircraft query method and system based on balanced hash coding. The method comprises the following steps: modeling and representing a corresponding aircraft by utilizing flight parameter data and flight condition data of each aircraft in a data set; generating a corresponding vector space for each coded aircraft to limit the balance and identifiability of hash code generation, and determining a hash code mode of the aircraft; performing hash coding on the target aircraft by using the determined hash coding mode; and carrying out matching query by utilizing hash codeinformation of the aircraft to finally obtain the aircraft matched with the target aircraft in the data set. An unsupervised learning mode without manual intervention is adopted to carry out balancedhash coding on the aircraft, so that the storage cost of the aircraft is effectively reduced, and retrieval operation is carried out based on the coded aircraft.

A Hash join method, device and database management system, the method comprising: when dividing a target data group during database query, using vector as a unit of quantity to divide and calculate the Hash value of the original data in a data segment, and representing the Hash value in bits; dividing the original data corresponding to the same Hash value of specified bits into the same group based on a preset grouping rule in Hash grouping, continuing to execute Hash grouping in subsequent grouping by utilizing the unspecified bits in the previous Hash grouping, and in the grouping process, according to the positions of the original data in the target data group, ranking the original data in the same group; and conducting a join operation on the grouped and ranked original data to be joined in the corresponding groups in the target data group, thus reducing the complexity of subsequent ranking of each group.