Industrial manufacturing real-time stream processing method, equipment and medium

By splitting message queues according to business theme identifiers, extracting dimension keys and time identifiers to form tuples, and performing hash partitioning, the problem of out-of-order and backlog issues in real-time streaming data processing in industrial manufacturing is solved, improving the efficiency and accuracy of multi-threaded concurrent computing.

CN121029441AActive Publication Date: 2025-11-28INSPUR YUNZHOU (SHANDONG) IND INTERNET CO LTD
6 Cites 0 Cited by

Patent Information

Application Number
CN202511575823.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-10-31
Publication Date
2025-11-28
Estimated Expiration
2045-10-31

AI Technical Summary

Technical Problem

Existing real-time streaming data processing methods are prone to data time disorder and message backlog when multi-threaded concurrent computing, and single-threaded computing efficiency is low, especially when the data volume is huge in industrial manufacturing.

Method used

By splitting the original message queue according to the business theme identifier, dividing it into batches by time window, and extracting the dimension key, unique key and time identifier to form tuples, and using the group identifier for hash partitioning, we can ensure that the data is processed in chronological order and avoid out-of-order and message backlog.

Benefits of technology

It ensures the time order of data in multi-threaded concurrent computing, improves the efficiency of real-time streaming data processing, avoids message backlog and computation delay, and adapts to efficient processing of different data volume scenarios.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses an industrial manufacturing real-time stream processing method and device and a medium, relates to the field of data processing, and solves the problems of data time disorder and message accumulation. The method comprises the following steps: splitting service data into corresponding sub-queues according to a service theme identification field; and dividing the sub-queues into batches with preset specifications according to a preset time window. For each piece of data of each batch, extracting a dimension key, a unique key and a time identifier, and packaging the dimension key, the unique key and the time identifier into a first tuple; and generating a data group according to the dimension key grouping, splicing the group identifier of each group of unique keys according to a preset rule, and packaging the group identifier and the data group into a second tuple. And performing hash partitioning on the second tuple according to the group identifier, splitting the second tuple of each partition into single data, and calculating an index according to a time sequence. According to the method, the message queues are split according to the identifier combination, the three-dimensional identifier groups such as the dimension keys are extracted, and the unique key hash partitions are spliced, so that the real-time streaming data are processed according to the time sequence, meanwhile, real-time index calculation is efficiently completed, and the processing efficiency is improved.
Need to check novelty before this filing date? Find Prior Art