Unlock instant, AI-driven research and patent intelligence for your innovation.

Data processing method and device, and electronic equipment

A data processing and data set technology, applied in the computer field, can solve problems such as low data processing efficiency, and achieve the effects of reducing resource consumption, reducing data delay, and improving data processing efficiency

Pending Publication Date: 2022-02-25
CHINA MOBILE GRP GUANGDONG CO LTD +1
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The purpose of the embodiments of the present invention is to provide a data processing method, device and electronic equipment to solve the problem of low data processing efficiency in the prior art when data distribution and loading are performed

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device, and electronic equipment
  • Data processing method and device, and electronic equipment
  • Data processing method and device, and electronic equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0020] Such as figure 1 As shown, the embodiment of the present invention provides a data processing method. The execution subject of the method may be a server, and the server may be an independent server or a server cluster composed of multiple servers. The method specifically may include the following steps:

[0021] In S102, based on a preset data interface, target data for a target service is collected.

[0022] Wherein, the target service may be any mobile communication service, the target data may include signaling data and / or user plane data for the target service, and the preset data interface may be any data interface that can obtain the target data, for example, preset data The interface may include a Kafka data interface, a preset file transfer protocol (File Transfer Protocol, FTP) interface, a WebService interface, and the like.

[0023] Among them, the terminal electronic device can be any electronic device held by the on-site staff and used to collect the bas...

Embodiment 2

[0041] Such as figure 2 As shown, the embodiment of the present invention provides a data processing method. The execution subject of the method may be a server, and the server may be an independent server or a server cluster composed of multiple servers. The method specifically may include the following steps:

[0042] In S202, the queue capacity of the lock-free queue is obtained, and based on the queue capacity, target data for the target service is collected through a preset data interface.

[0043] In the implementation, the target data is collected through the queue capacity of the lock-free queue, which can reduce the data interaction time and avoid the data waiting problem caused by the data volume of the target data being greater than the queue capacity of the lock-free queue.

[0044] A data collection thread can be defined, which is used to collect the target data for the target business through the preset data interface.

[0045] The lock-free queue can be used ...

Embodiment 3

[0103] The above is the data processing method provided by the embodiment of the present invention. Based on the same idea, the embodiment of the present invention also provides a data processing device, such as image 3 shown.

[0104] The data processing device includes: a data acquisition module 301, a data input module 302, a data set acquisition module 303 and a data processing module 304, wherein:

[0105] A data collection module 301, configured to collect target data for a target service based on a preset data interface, where the target data includes signaling data and / or user plane data for the target service;

[0106] The data input module 302 is configured to put the target data into a preset lock-free queue, and classify the target data based on preset classification rules and data characteristics of the target data to obtain multiple classified data sets , the lock-free queue is a queue constructed based on a ring array and does not need to be locked;

[0107] ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention discloses a data processing method and device, and electronic equipment. The method comprises the following steps: collecting target data for a target service based on a preset data interface, wherein the target data comprise signaling data and / or user plane data for the target service; putting the target data into a preset lock-free queue, and classifying the target data based on a preset classification rule and the data features of the target data to obtain a plurality of classification data sets; based on the lock-free queue, acquiring a first data set meeting a preset calculation requirement and a second data set meeting a preset loading requirement in the classified data set; and based on the lock-free queue, distributing the target data in the first data set to a preset message publishing system, and loading the target data in the second data set to a preset distributed file system. Through the method, the data processing efficiency during data distribution and loading can be improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a data processing method, device and electronic equipment. Background technique [0002] With the continuous development of computer technology, the number of users of mobile communication services has increased sharply. How to meet the calculation requirements (such as real-time calculation and offline calculation, etc.) of mobile communication service data with a large amount of data has become the focus of attention of operators. [0003] At present, the distribution of multi-channel data streams can be carried out through the double-ended blocking queue based on the linked list of the log server, that is, the mobile communication service data is pushed to the message publishing system (such as Kafka) through the log server, and at the same time, the log server can also send The mobile communication service data is loaded into the distributed file system (Hadoop Dist...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/54G06F16/182G06K9/62
CPCG06F9/546G06F16/182G06F2209/548G06F18/24
Inventor 张晓川谭丽丽张湛梅肖文彬谢梦
Owner CHINA MOBILE GRP GUANGDONG CO LTD