High-dimensional feature processing method and device

A processing method and a technology of a processing device, which are applied in special data processing applications, electrical digital data processing, character and pattern recognition, etc., can solve problems such as failure to achieve a high recall rate, and reduce computing costs and time. Easy to scale and expand, improve the effect of recall rate

Pending Publication Date: 2020-03-24
SHENZHEN ZTE NETVIEW TECH +1
View PDF5 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

At present, the high-dimensional feature storage index scheme based on product quantization or HNSW scheme design alone has been applied, but it does not p

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • High-dimensional feature processing method and device
  • High-dimensional feature processing method and device
  • High-dimensional feature processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0044] like figure 1 As shown, the processing method of the high-dimensional features of the present application, an implementation thereof, comprises the following steps:

[0045] Step 102: Select sample features, perform rough quantization on the sample features, and generate multiple cluster centers. The sample features in this application refer to high-dimensional features as samples.

[0046] Step 104: Add the high-dimensional features to be inserted into the corresponding HNSW (Hierarchcal Navigable Small World graphs) clusters according to the principle of the closest distance.

[0047] Step 106: Calculate the distance between the target feature and the cluster center, perform HNSW algorithm retrieval in the preset number of clusters closest to the sorting distance, and return the retrieval result. The target features in this application refer to the high-dimensional features to be retrieved.

[0048] HNSW is an optimized version of NSW (Navigable Small World graphs,...

Embodiment 2

[0065] like Image 6 As shown, the high-dimensional feature processing device 600 of the present application, an implementation manner thereof, may include a rough quantization module 610 , a clustering module 620 and a retrieval module 630 .

[0066] The coarse quantization module 610 is used to select sample features, perform rough quantization on sample features, and generate multiple cluster centers;

[0067] The clustering module 620 is used to add the high-dimensional features to the corresponding HNSW clusters according to the principle of the closest distance;

[0068] The retrieval module 630 is used to calculate the distance between the target feature and the cluster center, perform retrieval among the preset number of clusters closest to the sorting distance, and return the retrieval result.

[0069] like Figure 7 As shown, another embodiment of the high-dimensional feature processing device 700 of the present application may include a coarse quantization module,...

Embodiment 3

[0079] An embodiment of the high-dimensional feature processing device of the present application includes a memory and a processor.

[0080] memory for storing programs;

[0081] The processor is configured to implement the method in Embodiment 1 by executing the program stored in the memory.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a high-dimensional feature processing method and device, and the method comprises the steps: selecting a sample feature, carrying out the coarse quantification of the sample feature, and generating a plurality of clustering centers; respectively adding the high-dimensional features to be inserted into the corresponding HNSW clusters according to the principle of closest distance; and respectively calculating the distances between the target features and the clustering centers, performing HNSW algorithm retrieval in a preset number of clusters closest to the sequence, and returning a retrieval result. In the embodiment of the invention, clustering is carried out by using coarse quantization, so that the method is suitable for large-scale distributed fragmentation storage and is easy for expansion and contraction of mass storage; due to the fact that HNSW algorithm retrieval is carried out in the preset number of clusters closest to the sequence, the recall rate is increased; and, according to the method, the advantages of vector quantization and HNSW are combined at the same time, and the calculation cost and time of data insertion and retrieval are reduced.

Description

technical field [0001] The present application relates to image retrieval, in particular to a method and device for processing high-dimensional features. Background technique [0002] At present, deep learning has been deeply applied in many fields, especially in face recognition related application scenarios. The public security department of a large city collects real-time video through a large number of deployed video surveillance systems, and then the high-dimensional features of faces extracted by the face recognition system can reach tens of millions per day, and billions per year. At present, the high-dimensional feature storage index scheme based on product quantization or HNSW scheme design alone has been applied, but it does not perform well in the search of billion-level feature databases. It is almost impossible to perform queries at the second level and maintain a high recall rate. The existing technology cannot achieve a high recall rate while completing secon...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/583G06F16/55G06K9/00G06K9/62
CPCG06F16/583G06F16/55G06V40/168G06F18/23213
Inventor 陈晓东朱金华
Owner SHENZHEN ZTE NETVIEW TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap