25 results about "Locality-sensitive hashing" patented technology

The invention discloses a large-scale embedded index data desensitization method suitable for a forestry database, and belongs to the technical field of data desensitization, and the method comprises the steps: obtaining a forestry data description document D1, splitting the document D1 into sentences, and processing the sentences into embedded vectors; a user customizes a query sentence containing the desensitization word, and a query vector is generated; performing hash coding on the embedded vector and the query vector to obtain a corresponding hash function cluster; searching data to be desensitized through cosine similarity calculation; and inputting to-be-desensitized data and the selected desensitization method into the large language model to generate desensitized data, and replacing the corresponding sentences in the D1 to obtain a desensitized forestry data description document D2. According to the method, the input data is converted into the high-dimensional vector through the embedding layer, so that the semantic features of the data can be effectively expressed in the vector space, the retrieval speed and accuracy of large-scale data are improved by means of the locality sensitive hashing technology, the calculation overhead is reduced, and a technical guarantee is provided for real-time desensitization processing of a large-scale data set.

A system for rideshare matching using locality sensitive hashing is disclosed, including at least one rider device and at least one driver device in operable connection with a network. A rideshare application is in operable communication with the network and configured for matching a driver to a rider within a match pool via an artificial intelligence engine operating a locality sensitive hashing module.

The invention provides a smart city lighting advertisement management method and system, and the method comprises the steps: collecting the state data of a lighting device, an advertisement resource and an environment scene through a multi-source sensing network, and carrying out the preprocessing and fusion through an edge calculation node, and obtaining a standardized data flow; based on the standardized data stream, the lighting equipment and the advertisement resources are subjected to bucket separation by adopting locality sensitive hashing, optimization is carried out through iterative local search, and a resource grouping result is obtained; constructing a random regular graph model, optimizing a graph structure through a Markov chain Monte Carlo algorithm, extracting a mapping relationship between the lighting equipment and the advertisement resources, and generating a resource allocation strategy; generating a cooperative control strategy through a multi-objective optimization algorithm on the basis of a resource configuration strategy in combination with the user behavior data and the state data of the environment scene; and based on the cooperative control strategy, controlling the lighting equipment and the advertisement resources and collecting feedback data. According to the invention, intelligent collaborative management of urban lighting and advertisement resources is realized.

Strings of a text file representing a configuration of a target device are respectively tokenized into tokens for the configuration. The tokens for the configuration are shingled. A target device signature representing the configuration of the target device is generated by applying a min-wise independent permutations locality sensitive hashing (MinHash) technique to the tokens as have been shingled. Whether the configuration of the target device is anomalous is identified based on the target device signature.

The invention provides a multi-target accurate retrieval method and system in a security video, and relates to the field of interdisciplinary application, and the method is characterized in that the method comprises the following steps: carrying out the standardized preprocessing of an original security video, and deploying an improved YOLOv8 model on a preprocessed key frame to carry out the high-precision multi-target detection. The method has the advantages that the appearance, motion and semantic complementary features of the target are synchronously extracted by improving the YOLOv8 model, cross-frame association and trajectory modeling are realized by adopting an enhanced DeepSORT tracker, and a compact feature index database and a three-level progressive retrieval mechanism are constructed in combination with a locality sensitive hashing algorithm; according to the method, the multi-target retrieval precision and efficiency in massive videos are remarkably improved, the target relevance and the complex scene semantic understanding ability are enhanced, meanwhile, the calculation overhead is greatly reduced through rapid approximate matching and a hierarchical retrieval strategy, and the requirements of security application for high accuracy, rapid response and real-time intelligent research and judgment are met.

The invention provides an NL2SQL system based on vector embedding and locality sensitive hashing. The NL2SQL system comprises a data acquisition module, a preprocessing module, a list screening module, an SQL generation module, a result verification module and a cache management module. Collecting a natural language query problem; performing multi-modal integration processing on the data in the database to obtain corresponding multi-modal data, and performing vector conversion processing to obtain a multi-modal data vector table and a multi-modal data vector column; constructing a locality sensitive hash index; performing feature processing on the natural language query problem to obtain query feature data, performing vector conversion processing to obtain a query feature vector, further obtaining a query multi-modal data vector table and a query multi-modal data vector column, and generating an SQL query statement; verifying the SQL query statement to obtain a query verification result, and generating an execution error processing scheme; and the accuracy and the efficiency of the NL2SQL system are improved.

The invention discloses a digital media processing method and system based on an improved channel pruning algorithm, and relates to the technical field of digital media processing, and the method comprises the steps: obtaining a target digital media; digital media data to be processed are processed through a CNN convolutional neural network improved by a channel pruning algorithm based on locality sensitive hashing to obtain target information, and the channel pruning algorithm comprises a block division stage, a block selection stage and a fine tuning stage. According to the method, the relevance of input and output channels is kept based on the locality sensitive hash pruning algorithm, the candidate space of channel selection is expanded, the to-be-pruned channels can be rapidly screened, the high calculation cost of solving the sparsification problem through multiple iterations in a traditional method is avoided, the pruning efficiency is remarkably improved, and the method is suitable for large-scale popularization and application. The advantages of structured pruning are reserved while the blocks are efficiently selected, the digital media processing speed is improved on the premise that the performance requirement of digital media is maintained, and the memory is reduced, so that the method is more widely applied to processors of different specifications.

The present application relates to the technical field of image retrieval, and particularly to an image retrieval method based on a combination of a deep convolutional neural network and a local sensitive hashing algorithm, which has the following steps: step S1: training set and validation set in an open source data set of image retrieval; step S2: input of the model during training; step S3: test retrieval ranking; step S4: the loss function of image retrieval adopts a contrast loss function, and in addition to mAP, the model evaluation index also newly adds mP@k; the present method is based on a combination of a deep convolutional neural network and a local sensitive hashing algorithm, the algorithm extracts image features of a gallery library and a query library in a deep convolutional manner, carries out LSH hash coding, greatly improves the retrieval performance, and uses contrast learning in a twin network, thereby greatly improving the retrieval precision.

In implementations of systems for generating indications of relationships between electronic documents, a processing device implements a relationship system to segment text of electronic documents included in a document corpus into segments. The relationship system determines a subset of the electronic documents that includes electronic document pairs having a number of similar segments that is greater than a threshold number. The similar segments are identified using locality sensitive hashing. The electronic document pairs are classified as related documents or unrelated documents using a machine learning model that receives a pair of electronic documents as an input and generates an indication of a classification for the pair of electronic documents as an output. Indications of relationships between particular electronic documents included in the subset are generated based at least partially on the electronic document pairs that are classified as related documents.

The application discloses a kind of local sensitive hash-based internet of things equipment identification method: (1) obtaining device traffic at gateway, and classifying according to the physical address of specific device;(2) the device traffic of good classification is filtered, slicing and reorganization to create device signature;(3) using local sensitive hash algorithm to generate digest for each signature, digest is stored in digest database together with device label;(4) generate the digest of the traffic to be identified, compare it with all instances in the database, calculate four indexes;(5) use the three linear regression model trained by multi-index evaluation method to fit the similarity score of 10 pieces of signature aggregation, and the device with the highest fitting score is returned as the predicted value of the device.The application can realize fast and accurate fine-grained internet of things equipment identification, and avoids the cumbersome process of feature extraction and model training in machine learning, reduces the computing overhead and is simple to deploy.

Techniques for verifying correctness of associations between assets related to events detected in at least one computer network and assets in an asset catalog for the at least one computer network. The techniques include obtaining information specifying a first asset and a first set of assets with which the first asset was previously associated; generating a signature of the first asset from the computer network addressing information for the first asset; generating a hashed signature by applying a locality sensitive hashing (LSH) technique to the signature; associating the first asset with a second set of assets in the asset catalog using the hashed signature and at least one hashed signature of the at least one asset in the asset catalog; and when it is determined that the second set of includes the first set, outputting an indication that the first asset was correctly associated with the first set of assets.

The invention provides a deep neural network reasoning acceleration method and system, and relates to the technical field of deep learning, and the method comprises the steps: constructing an initial reasoning acceleration model based on a preset data set; the sparse degree of neurons of any layer in the initial reasoning acceleration model is detected according to the locality sensitive hashing technology, and the activation threshold value of the neurons is dynamically adjusted according to the sparse degree; on the basis of a Bayesian optimization algorithm, pruning operation is carried out on the initial reasoning acceleration model after the activation threshold value is dynamically adjusted, and a target reasoning acceleration model is obtained; and executing an image classification task by using the target reasoning acceleration model so as to output a reasoning result through the target reasoning acceleration model. According to the method, by dynamically adjusting the neuron activation threshold and the hierarchical pruning strategy and combining the locality sensitive hashing technology and Bayesian optimization, efficient reasoning acceleration of the deep neural network on the edge computing device is achieved, meanwhile, the influence on the model precision is minimized, and the reasoning performance and practicability of the model under the low-power-consumption condition are improved.

This application provides a method for detecting accompanying domain names based on the Locality Sensitive Hashing (LSH) algorithm, which includes the following steps: calculating spatiotemporal accompanying pairs using LSH, and filtering out intersecting spatiotemporal accompanying pairs; calculating and sorting the true C-Rank scores of the intersecting spatiotemporal accompanying pairs; and grouping the target domain names, taking the accompanying pairs of target domain names with a similarity greater than a similarity threshold as the final spatiotemporal accompanying pairs. In industrial-scale massive data scenarios, this application, based on domain name request accompanying relationships, can quickly calculate domain name accompanying pairs using LSH in linear time complexity.

Systems and methods for counteracting data-skewness for locality sensitive hashing via feature selection and pruning are disclosed. In one embodiment, a method for feature selection for counteracting data skewness on locality sensitive hashing (LSH)-based search may include: (1) ingesting, by an ingestion computer program and from a plurality of data sources, data; (2) extracting, by the ingestion computer program, a plurality of features from the ingested data; (3) transforming, by the ingestion computer program, each of the plurality of features into a feature vector; (4) selecting, by the ingestion computer program, a subset of the plurality of features; and (5) for each selected feature vector: computing, by the ingestion computer program, a random hash function for the selected feature; and inserting, by the ingestion computer program, an output of the random hash function into a hash table with the selected feature.

Proposed is a digital system using Locality Sensitive Hashing (LSH) for near duplicate detection on weighted datasets providing a robust technical solution to the problem by improving (i) efficiency by allowing for fast and accurate identification of near duplicates in large datasets; (ii) accuracy by extending the hashes calculated via 5 the LSH approach with an insurance-specific weight, thus making sure that the concept of similarity between documents is calculated on a technical and industry-relevant basis; (iii) cost reduction by reducing redundant data lowers storage and processing costs; (iv) data integrity by improved accuracy and consistency in records enhance overall data integrity; and (v) regulatory compliance matching by automated data 10 management practices supporting compliance with industry regulations.

Techniques for associating assets related to events detected in at least one computer network with respective assets in an asset catalog for the at least one computer network. The techniques include: while monitoring activity on the at least one computer network, obtaining information about an event related to a first asset, the information specifying computer network addressing information for the first asset; generating a signature of the first asset from the computer network addressing information; generating a hashed signature of the first asset by applying a locality sensitive hashing (LSH) technique to the signature; associating the first asset with at least one asset in the asset catalog using the hashed signature of the first asset and at least one hashed signature of the at least one asset in the asset catalog; and outputting information identifying the at least one asset with which the first asset was associated.

Systems and methods for counteracting data-skewness for locality sensitive hashing via feature selection and pruning are disclosed. In one embodiment, a method for feature selection for counteracting data skewness on locality sensitive hashing (LSH)-based search may include: (1) ingesting, by an ingestion computer program and from a plurality of data sources, data; (2) extracting, by the ingestion computer program, a plurality of features from the ingested data; (3) transforming, by the ingestion computer program, each of the plurality of features into a feature vector; (4) selecting, by the ingestion computer program, a subset of the plurality of features; and (5) for each selected feature vector: computing, by the ingestion computer program, a random hash function for the selected feature; and inserting, by the ingestion computer program, an output of the random hash function into a hash table with the selected feature.

The invention discloses a black box adversarial text generation method based on semantic clustering and gradient fusion, which belongs to the field of computer security, and comprises the following steps: generating an initial adversarial text of an original text based on synonym replacement; constructing a candidate text list P based on locality sensitive hashing and semantic clustering; selecting a current optimal text xbest from the P; sampling a plurality of candidate text pairs from the P; processing each candidate text pair based on gradient fusion to generate candidate confrontation texts, and constructing a first candidate list P1 together with xbest; and generating an optimal adversarial text x * based on semantic clustering. According to the method, in a black box scene with hard tag setting, the query efficiency is greatly improved under the condition of keeping the confrontation text quality and semantic consistency, so that the requirement of verifying the robustness of an existing natural language processing model in confrontation of confrontation attacks in a real scene with limited query times is met.

A system and method for identifying a speaker based on dual calculation of voiceprint match scores, using machine learned voiceprint sets and machine learned hashing models to reduce a voiceprint set, normalizing the joint scores, and performing user identification. The system and method include, among other things, a novel normalization technique designed for multitarget detection that considers scores between all enrolled negative list (NL) utterances and the normalization cohort as a single distribution. The system and method include machine learned hashing models to efficiently find a small subset of utterances from enrolled NL utterances and the normalization cohort that are most similar to the test utterance, so that the number of similarity score computations can be significantly reduced in identifying the speaker.

The application relates to a file tree matching approximate retrieval method and system based on a local sensitive hashing algorithm, which comprises the following steps: calculating the sha1 values of sub-file trees and sub-files under a file tree through a local sensitive hashing algorithm, and generating a hash vector of the file tree; judging the similarity between file trees by calculating the distance between the hash vectors of two file trees; performing multiple clustering on the hash values of the file trees through a hierarchical clustering algorithm; in the search, starting from the clustering center point of the highest layer, comparing to obtain the nearest point, then searching the nearest child node under the point, and traversing all nodes in the lowest layer to return the top-k most similar file tree nodes. According to the application, a user inputs a file tree path, the program is parsed, the local sensitive hash value of the file tree is calculated while the traditional hash signature is generated, then the hierarchical clustering algorithm is used to perform multiple-layer clustering on the local sensitive hash value, and in the retrieval, the similar open source component library is quickly obtained through the clustering center nodes of each layer.

A data item identification service may use locality sensitive hashing to identify data items which are relevant to a search based on a hash of a search vector based on the search. Hashes of vectors based on data items may be stored using a set of bitmaps, where a given bitmap corresponds to a given bit position of a hash. Data items with matching or similar hashes may be related to each other. The data item identification service may perform searches by generating a hash for a search vector and comparing the hash to hashes for data items using one or more logical operations on the set of bitmaps to identify related data items.

The invention relates to the technical field of optical processing energy allocation, and discloses a point cloud topology perception-based fine structure self-adaptive polishing method, which comprises the following steps of: constructing an edge region geometric topology curvature tensor field by using in-situ white light interference point cloud data, and extracting a feature vector with geometric invariance; through a locality sensitive hashing algorithm, matching an energy allocation parameter containing the amplitude of the output current of the power supply adjusting circuit in the retrieval matrix, determining the power load pulse width required by the power supply adjusting circuit for driving the polishing load system, and controlling the polishing load system to operate along the Pierino fractal path, electric energy precise allocation is achieved through the power supply adjusting circuit, edge pressure nonlinear distribution is compensated, machining errors generated by boundary condition truncation are eliminated, materials are prevented from being excessively removed, coherence superposition of machining residual textures is damaged, and the conformal precision and the surface quality of the edge area of the precise optical assembly are effectively improved.

The invention discloses a short-term photovoltaic power prediction processing method, system and platform based on reversible normalization and day and night coding, and the method comprises the steps: obtaining historical meteorological data and historical photovoltaic power data of a photovoltaic station, and carrying out the data preprocessing; dividing a training set, a verification set and a test set according to a preset proportion; for each batch of examples in a training set, a verification set and a test set, a reversible instance normalization technology is introduced to effectively alleviate a data distribution offset problem, and a periodic rule of photovoltaic power generation is deeply fused by using refined day and night time coding. And the long sequence processing efficiency is improved by adopting an efficient attention mechanism based on locality sensitive hashing, so that the precision and the stability of short-term photovoltaic power prediction are remarkably improved.

This invention discloses an image retrieval method based on the Locality Sensitive Hash (LSH) algorithm, comprising the following steps: Step 1, acquiring multiple image data, preprocessing the multiple image data to obtain a densely represented image dataset A; using a d-dimensional vector x (x∈A) to represent one image data in the image dataset A, x=[x1,x2,...,x... i ,...x d Step 2: Based on image dataset A, initialize the SoftHash parameters and update them based on the image data; where the SoftHash parameters include: synaptic weight matrix W and neuron bias vector b; Step 3: Given an image s with dense representation, use the synaptic weight matrix W and neuron bias vector b obtained in Step 2 to generate a binary representation v of image s with hash length k; Step 4: Based on the obtained binary representation v of image s with hash length k, perform fast nearest neighbor retrieval using Hamming distance to find images with the same object instance as the query image.