Distributed comparison clustering method and device, electronic equipment and storage medium

A clustering method and distributed technology, applied in the field of data clustering, can solve the problems of low data calculation efficiency, huge amount of monitoring image data, and inability to fully utilize the computing power of each node.

Active Publication Date: 2019-11-29
PCI TECH GRP CO LTD
View PDF13 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

At present, traditional data comparison and analysis mostly use a single node for the calculation of comparison and clustering tasks, but with the increase in the size of feature data, single-node comparison and clustering can no longer meet the computing needs of products and projects
For urban security monitoring, the amount of monitoring image data is huge, and only a single node is used to perform comparison and clustering tasks, the CPU load is too large, and the data calculation efficiency is...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed comparison clustering method and device, electronic equipment and storage medium
  • Distributed comparison clustering method and device, electronic equipment and storage medium
  • Distributed comparison clustering method and device, electronic equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 2

[0072] On the basis of the above examples, Figure 8 It is a flow chart of another distributed alignment and clustering method provided in Embodiment 2 of the present application. Correspondingly applied to the management node, refer to Figure 8 , the distributed comparison and clustering method provided in this embodiment specifically includes:

[0073] S210. Receive a comparison and clustering task, and divide the comparison and clustering task into several subtasks;

[0074] S220. Put the subtasks into a task queue for each computing node to receive and process them one by one;

[0075] S230. Obtain the comparison and clustering results corresponding to each subtask sent by each computing node from the result queue;

[0076] S240. Perform clustering on the comparison and clustering results of each of the subtasks, generate a clustering result corresponding to the comparison and clustering task, and send it back to the task requester of the comparison and clustering task...

Embodiment 3

[0080] On the basis of the above examples, Figure 9 It is a schematic structural diagram of a distributed comparison and clustering device provided in Embodiment 3 of the present application. refer to Figure 9 , the distributed comparison and clustering device provided in this embodiment specifically includes: a task receiving module 31 , a comparison and clustering module 32 and a new task processing module 33 .

[0081] Wherein, the task receiving module 31 is used to monitor the task queue in real time, receive subtasks from the task queue, and the subtasks are received by the management node for comparison and clustering tasks, and the comparison and clustering tasks are equally divided into several subtasks. The subtasks are put into the task queue; the comparison and clustering module 32 is used to obtain the characteristic data set corresponding to the comparison and clustering task from the management node, and process the subtasks based on the characteristic data s...

Embodiment 4

[0087] Embodiment 4 of the present application provides an electronic device, and the electronic device includes: a processor, a memory, and a communication module. The number of processors in the electronic device may be one or more, and the number of memories in the electronic device may be one or more. The processor, memory, and communication module of the electronic device may be connected through a bus or in other ways.

[0088] As a computer-readable storage medium, memory can be used to store software programs, computer-executable programs and modules, such as program instructions / modules corresponding to the distributed comparison and clustering method described in any embodiment of the present application (for example, distributed Comparing the task receiving module in the clustering device, comparing the clustering module and the new task processing module). The memory may mainly include a program storage area and a data storage area, wherein the program storage are...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention discloses a distributed comparison clustering method and a device, electronic equipment and a storage medium. The method according to an embodiment of the invention comprises the following steps: receiving sub-tasks equally divided by clustering tasks in a task queue through each computing node; processing the received sub-tasks by the computing node; and sending acomparison clustering result generated by processing to a result queue, continuously getting new sub-tasks from the task queue for processing until all the sub-tasks equally divided corresponding to one clustering task are processed, and finally summarizing the comparison clustering result of clustering each sub-task by the management node to finish the comparison clustering task. And the computing nodes process one subtask according to the processing process of the subtask and then get a new subtask until the comparison clustering task is completed, so that the computing power of each computing node is balanced and fully utilized, and the data comparison clustering is more efficient.

Description

technical field [0001] The embodiments of the present application relate to the technical field of data clustering, and in particular, to a distributed comparison and clustering method, device, electronic equipment, and storage medium. Background technique [0002] In the security tracking and monitoring scenario, it is necessary to compare and cluster the video images monitored by the camera, and use the image feature data after comparison and clustering for subsequent security data analysis. At present, traditional data comparison and analysis mostly use a single node to perform comparison and clustering tasks. However, with the increase in the size of feature data, single-node comparison and clustering can no longer meet the computing needs of products and projects. For urban security monitoring, the amount of monitoring image data is huge, and only a single node is used to perform comparison and clustering tasks. The CPU load is too large and the data calculation efficie...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/62G06F9/50
CPCG06F9/5038G06F18/23Y02D10/00
Inventor 李博郑轩廖海贾志忠郑娃龙
Owner PCI TECH GRP CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap