Data processing method, device and electronic equipment

Abstract

Embodiments of the present invention disclose a data processing method, device, and electronic device. The method includes: based on a preset data interface, collecting real-time conversation messages and target offline files generated during the processing of a communication service, wherein the target offline files are used to settle the communication service during a preset settlement period; performing preset message conversion on the target offline files, converting the target offline files into target offline messages, and placing the real-time conversation messages and the target offline messages into a first message queue to be processed; based on a preset distributed data processing service, processing the messages in the first message queue to be processed, and placing the processed messages into a second message queue to be processed, so as to process the communication service based on the messages in the second message queue to be processed. This method can improve the data processing efficiency of the communication service.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data processing method, device and electronic equipment. Background Art

[0002] With the continuous development of computer technology, operators can provide more and more communication services to users. However, different communication services have different settlement methods. As the number of users continues to grow, the amount of business that operators need to handle is relatively large. Therefore, how to improve business processing efficiency has become the focus of operators.

[0003] For example, operators can perform online billing based on real-time conversation messages generated by users during their communication service usage, or they can perform offline billing based on the call record files generated by users during a predetermined settlement period (such as a month). Currently, operators can process real-time conversation messages through shared storage for online billing, and can also collect and process user call record files through the messaging system for offline billing.

[0004] However, due to the shared storage method, real-time conversation messages need to be frequently delivered when processing data, resulting in low data processing efficiency. The message system cannot process real-time conversation messages due to the high response delay problem of synchronous sending and receiving messages. Therefore, when processing communication services, using the above two methods to handle different communication services separately will result in low data processing efficiency. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a data processing method, device and electronic device to solve the problem of low data processing efficiency in processing communication services in the prior art.

[0006] To solve the above technical problems, the embodiments of the present invention are implemented as follows:

[0007] In a first aspect, an embodiment of the present invention provides a data processing method, the method comprising: based on a preset data interface, collecting real-time conversation messages and target offline files generated during communication service processing, the target offline files being used to perform settlement processing on the communication service in a preset settlement period; performing preset message conversion processing on the target offline files, converting the target offline files into target offline messages, and placing the real-time conversation messages and the target offline messages into a first message queue to be processed; based on a preset distributed data processing service, processing the messages in the first message queue to be processed, and placing the processed messages into a second message queue to be processed, so as to process the communication service based on the messages in the second message queue to be processed.

[0008] In a second aspect, an embodiment of the present invention provides a data processing device, comprising: a data acquisition module for collecting real-time conversation messages and target offline files generated during the communication service processing based on a preset data interface, wherein the target offline file is used to perform settlement processing on the communication service in a preset settlement period; a first processing module for performing preset message processing on the target offline file, converting the target offline file into a target offline message, and placing the real-time conversation message and the target offline message into a first message queue to be processed; a second processing module for processing the messages in the first message queue to be processed based on a preset distributed data processing service, and placing the processed messages into a second message queue to be processed, so as to process the communication service based on the messages in the second message queue to be processed.

[0009] In a third aspect, an embodiment of the present invention provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the data processing method provided in the above embodiment.

[0010] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the data processing method provided in the above embodiment are implemented.

[0011] As can be seen from the technical solutions provided by the above embodiments of the present invention, the embodiments of the present invention collect real-time conversation messages and target offline files generated during the communication service processing based on a preset data interface. The target offline files are used to settle the communication service in a preset settlement period. The target offline files are processed into preset messages, and the target offline files are converted into target offline messages. The real-time conversation messages and the target offline messages are placed in a first message queue to be processed. Based on a preset distributed data processing service, the messages in the first message queue to be processed are processed and the processed messages are placed in a second message queue to be processed, so that the communication service is processed based on the messages in the second message queue to be processed. In this way, processing real-time conversation messages through a distributed data processing service can avoid the problem of low data processing efficiency caused by the need to frequently land real-time conversation messages in the shared storage data processing method. In addition, based on the present invention, real-time conversation messages and target offline files can be processed simultaneously, avoiding the problem of low data processing efficiency caused by using two methods (i.e., shared storage data processing method and message system data processing method) to process different data (i.e., real-time conversation messages and target offline files) separately, thereby improving data processing efficiency when processing communication services. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 A flow chart of a data processing method of the present invention;

[0014] Figure 2 A schematic diagram of a data processing method of the present invention;

[0015] Figure 3 A flow chart of another data processing method of the present invention;

[0016] Figure 4 is a schematic diagram of another data processing method of the present invention;

[0017] Figure 5 is a schematic diagram of another data processing method of the present invention;

[0018] Figure 6 This is a structural diagram of a data processing device according to the present invention;

[0019] Figure 7 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0020] Embodiments of the present invention provide a data processing method, device, and electronic device.

[0021] In order to enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0022] Example 1

[0023] like Figure 1 As shown, an embodiment of the present invention provides a data processing method. The execution subject of the method can be a server. The server can be an independent server or a server cluster composed of multiple servers. The method can specifically include the following steps:

[0024] In S102 , based on a preset data interface, real-time conversation messages and target offline files generated during the communication service processing are collected.

[0025] Among them, the preset data interface can be any interface that can collect real-time conversation messages and target offline files from network element side equipment (such as base stations, etc.). The real-time conversation messages can be conversation messages generated when the user uses the communication service. For example, the real-time conversation messages can be traffic messages generated when the user uses the traffic service. The target offline file can be used to settle the communication service in a preset settlement period. For example, assuming that the settlement period is one month, the target offline file can be a file generated based on the user's use of the communication service in the past month (such as the call duration, traffic volume, etc. generated by the user in the past month), and the user's communication service in the past month can be settled based on the target offline file.

[0026] In practice, with the continuous development of computer technology, operators can provide more and more communication services to users, and different communication services have different settlement methods. As the number of users continues to grow, the amount of business that operators need to handle is relatively large. Therefore, how to improve business processing efficiency has become the focus of operators.

[0027] For example, an operator can perform online billing based on real-time conversation messages generated by users during the use of communication services, or can perform offline billing based on the call record files generated by users during a predetermined settlement period (such as one month). Currently, operators can process real-time conversation messages for online billing through shared storage, and can also collect users' call record files through the message system and process the call record files for offline billing. However, when processing data based on the shared storage method, real-time conversation messages need to be frequently delivered, resulting in low data processing efficiency, and the message system cannot process real-time conversation messages due to the high response delay problem of synchronous message sending and receiving. Therefore, when processing communication services, using the above two methods to process different communication services separately will result in low data processing efficiency. To this end, an embodiment of the present invention provides another implementation scheme, which may specifically include the following contents:

[0028] Since current operators can provide users with a variety of different types of communication services, and different communication services can adopt different settlement methods, when users use communication services, they may generate a variety of communication data for processing different communication services. These communication data can be collected and processed by different devices.

[0029] Therefore, to improve data storage and processing efficiency, real-time conversation messages and target offline files generated during the communication service processing can be collected simultaneously through multiple different data interfaces. For example, the server can obtain real-time conversation messages generated by users when using communication services from the base station, and can also simultaneously obtain target offline files generated by users during a preset settlement period from a preset service processing server. Alternatively, the server can collect real-time conversation messages and target offline files generated during the communication service processing from a preset application (such as a data aggregation application provided by the operator).

[0030] There are many methods for collecting data. The above provides an optional and feasible data collection method. In actual application scenarios, there can also be many different data collection methods, which may vary according to the actual application scenarios. The embodiments of the present invention do not make specific limitations on this.

[0031] In addition, the real-time conversation messages and target offline files generated during the communication service processing can be collected based on a preset data collection period, wherein the preset data collection period can be one day, three days, one week, etc.

[0032] In S104, preset message conversion is performed on the target offline file, the target offline file is converted into a target offline message, and the real-time conversation message and the target offline message are put into a first message queue to be processed.

[0033] During implementation, the target offline file can be converted into one or more messages of defined processing representations. For example, the target offline file can contain three pieces of data (such as voice data 1, voice data 2, and traffic data 1) generated by the user using the communication service in the past three days. In this case, the target offline file can be converted into three target offline messages, and these three target offline messages can be messages of defined processing representations.

[0034] After receiving the target offline message, the real-time conversation message and the target offline message can be placed in the first queue of pending messages. The real-time conversation message is processed in the same manner as the target offline message. Furthermore, the real-time conversation message and the target offline message can be placed in the first queue of pending messages in order based on their priorities (e.g., time priority based on timestamp, quantity priority based on data size, etc.).

[0035] In S106, based on the preset distributed data processing service, the messages in the first message queue to be processed are processed, and the processed messages are put into the second message queue to be processed, so as to process the communication service based on the messages in the second message queue to be processed.

[0036] Among them, the preset distributed data processing service can be one or more services that can process messages in the first message queue to be processed. For example, the preset distributed data service can be a service that can check for duplicate messages in the first message queue to be processed, or the preset distributed data service can also be a service that can perform abnormal screening and processing on messages in the first message queue to be processed, etc. In addition, multiple different distributed data processing services can be determined based on location information, service type of communication service (such as 4G service, 5G service, Internet of Things service, etc.), processing type of communication service (such as real-time online settlement processing, offline settlement processing, etc.), etc.

[0037] In implementation, for example, taking multiple distributed data processing services for performing abnormal screening and processing of messages determined based on location information as an example, the server can collect real-time conversation messages and target offline files generated by users in area 1 when using communication services in the past three days through a preset data interface. After the target offline files are processed into preset messages to obtain target offline messages, the real-time conversation messages and target offline messages can be placed in the first message queue to be processed.

[0038] Assume that the first message queue to be processed contains 10 real-time conversation messages (including 6 real-time conversation messages for 4G services and 4 real-time conversation messages for 5G services) and 5 target offline messages (including 3 target offline messages for 4G services and 2 target offline messages for 5G services). There can be 2 sub-servers in area 1, one for the exception screening service for 4G services and the other for 5G services. The exception screening service can be a service that performs exception screening on the content, format, etc. of the message to filter out messages with exceptions.

[0039] So, if Figure 2 As shown, the server can distribute the 15 messages contained in the first pending message queue to different sub-servers for exception screening based on the different business types of the messages. That is, based on the preset distributed data processing service, the messages in the first pending message queue are processed. After the exception screening, the messages without exceptions are placed in the second pending message queue, and then the communication business (such as settlement processing) is processed based on the messages in the second pending message queue.

[0040] The above is an example of a distributed data processing service that is determined based on location information and is used to perform abnormal screening and processing on messages. In actual application scenarios, there may be multiple methods for dividing distributed data processing services, service types, etc., which may vary depending on the actual application scenarios. The embodiments of the present invention do not make specific limitations on this.

[0041] An embodiment of the present invention provides a data processing method that, based on a preset data interface, collects real-time conversation messages and target offline files generated during the processing of communication services. The target offline files are used to settle the communication services during a preset settlement period. The target offline files are processed into preset messages, converted into target offline messages, and the real-time conversation messages and the target offline messages are placed in a first message queue to be processed. Based on a preset distributed data processing service, the messages in the first message queue to be processed are processed and placed in a second message queue to be processed, so that the communication services are processed based on the messages in the second message queue. In this way, processing real-time conversation messages through a distributed data processing service can avoid the low data processing efficiency problem of shared storage-based data processing methods due to the need to frequently land real-time conversation messages. In addition, based on this method, real-time conversation messages and target offline files can be processed simultaneously, avoiding the low data processing efficiency problem caused by using two methods (i.e., shared storage-based data processing methods and message system-based data processing methods) to process different data (i.e., real-time conversation messages and target offline files) separately, thereby improving data processing efficiency when processing communication services.

[0042] Example 2

[0043] like Figure 3 As shown, an embodiment of the present invention provides a data processing method. The execution subject of the method can be a server. The server can be an independent server or a server cluster composed of multiple servers. The method can specifically include the following steps:

[0044] In S302 , based on a preset data interface, real-time conversation messages and target offline files generated during the communication service processing are collected.

[0045] In S304, preset message conversion is performed on the target offline file, the target offline file is converted into a target offline message, and the real-time conversation message and the target offline message are placed into a first message queue to be processed.

[0046] The specific processing procedures of S302 to S304 can refer to the relevant contents of S102 to S104 in the above embodiment 1, which will not be repeated here.

[0047] In S306, based on the preset distributed data duplicate checking service, duplicate checking is performed on the messages in the first queue of messages to be processed, and the messages after duplicate checking are placed in the second queue of messages to be processed.

[0048] In practice, in actual applications, the processing methods of the above S306 can be varied. The following provides an optional implementation method. For details, please refer to the following steps 1 to 2:

[0049] Step 1: Send the messages in the first to-be-processed message queue to multiple distributed data duplication checking processing services respectively.

[0050] In practice, the distributed data duplication checking and processing service can avoid repeated processing of the same real-time conversation message and / or target offline message, thereby improving the accuracy of communication business processing.

[0051] In step 2, each distributed data duplicate checking service performs duplicate checking on the messages received in the first message queue to be processed based on the preset full database, and puts the duplicate checked messages into the second message queue to be processed.

[0052] Among them, the preset full database can be a distributed database composed of real-time conversation messages and / or target offline files obtained based on a preset storage period. For example, the preset full database can be an HBase database that stores real-time conversation messages and / or target offline files for nearly one year.

[0053] In implementation, each distributed data duplicate checking service can perform duplicate checking on received messages based on a preset full database, and put the duplicate checked messages into a second queue of messages to be processed.

[0054] In addition, the server can also put the messages after duplicate checking into the third message queue to be processed, and then calculate and process the messages in the third message queue to be processed based on the preset distributed data accounting service, and put the messages after accounting and processing into the second message queue to be processed.

[0055] The accounting process may be a process of pricing real-time conversation messages and / or target offline messages based on preset pricing rules, or the accounting process may be a process of calculating whether the real-time conversation messages and / or target offline messages exceed the user's package content, etc.

[0056] In addition, before performing duplicate checking on the messages in the first message queue to be processed based on the preset distributed data duplicate checking service, the messages in the first message queue to be processed can also be preprocessed according to the preset distributed data preprocessing service, and the preprocessed messages can be put back into the first message queue to be processed.

[0057] For example, the messages in the first queue to be processed can be sent to multiple different distributed data pre-processing services respectively. Each distributed data pre-processing service can perform pre-duplication checking on the messages in the first queue to be processed based on a first-level duplication checking memory database (such as a database composed of real-time conversation messages and / or target offline messages in the past three months), and put the pre-duplication checked messages back into the first queue to be processed message.

[0058] Then, the server can send the messages in the first queue of pending messages after pre-checking for duplicates to multiple distributed data checking for duplicates processing services respectively. Each distributed data checking for duplicates processing service performs checking for duplicates on the messages in the first queue of pending messages received based on a preset full database (such as a secondary database consisting of real-time conversation messages and / or target offline messages for the past year), and places the messages after checking for duplicates into a third queue of pending messages. The server can perform accounting processing on the messages in the third queue of pending messages based on a preset distributed data accounting service. After each distributed data accounting service performs accounting processing on the received messages, it can place the calculated messages into the second queue of pending messages.

[0059] After checking for duplicates in the first queue of pending messages, S308 to S312 can be continued, or S314 can be continued, that is, messages without duplicate order problems can be processed through S308 to S312, and abnormal messages with duplicate order problems can be processed through S314.

[0060] In S308 , the messages in the second to-be-processed message queue are placed into a target service processing queue corresponding to the service processing type of the communication service.

[0061] Among them, the business types of communication services can be various. For example, the business types of communication services can include cumulative processing services, real-time account consolidation services, and call record storage services. Among them, the cumulative processing services can be services for summarizing different types of data generated during the communication service processing process. For example, the cumulative processing services can be services for summarizing the voice data used by each user who has used the group account in the past week. The real-time account consolidation services can be services for consolidating and settling messages based on the business types corresponding to the messages. For example, the real-time account consolidation services can be services for consolidating and settling voice messages contained in the second pending message queue. The call record storage services can be services for storing messages in the second pending message object in a preset database for subsequent preset operations.

[0062] In S310 , a target business processing service corresponding to the target business processing queue is obtained.

[0063] In implementation, taking the communication service as an account activation service as an example, the target service processing queue may be a queue containing account activation messages, and the target service processing service corresponding to the target service queue may be the account activation service.

[0064] In S312 , based on the target business processing service, the message in the target business processing queue is processed.

[0065] In practice, when processing communication services, there are sometimes strict requirements on the timing of communication services. For example, for account activation services, there are strong timing requirements. Therefore, for communication services with high timing requirements, the following steps 1 and 2 can be used to handle them:

[0066] Step 1: Based on the target business processing service, generate a processing task corresponding to each message in the target business processing queue.

[0067] Step 2: Based on the processing time of the processing tasks, the processing tasks are sequentially placed into the corresponding business processing queues, so as to process the processing tasks contained in the business processing queues based on the processing time of the processing tasks.

[0068] During implementation, when processing the processing tasks contained in the business processing queue, the processing tasks in the business processing queue can be distributed to multiple different distributed task processing services. Each distributed task processing service can process the tasks in sequence according to the processing time of the received processing tasks to meet the timing requirements of task processing.

[0069] In S314, duplicate messages in the first to-be-processed message queue are put into an abnormal message queue, and the messages in the abnormal message queue are processed based on a preset abnormal message processing service.

[0070] In implementation, the exception message processing service can perform risk detection on messages in the exception message queue, store messages in the storage (exceptional message database), and other processing.

[0071] In practical applications, a data processing system that implements the aforementioned data processing steps can be built on a pre-defined distributed messaging platform (such as Pulsar). Based on this data processing system, data can be transferred and processed between applications using Pulsar's messaging suite. All data to be processed (including real-time conversation messages, target offline files, and other related data) can be converted into Pulsar messages by front-end applications. Based on the data processing characteristics and business logic requirements of each link in the data flow, different subscriptions can be generated in Pulsar, and different subscriptions can be processed by different Subscription modes.

[0072] For example, a data processing system can be built based on Pulsar. First, the real-time conversation messages and target offline files on the network element side can be collected through the preset data interface. Then, the target offline files can be processed into preset messages through the data conversion module, and the target offline files can be converted into target offline messages.

[0073] like Figure 4 As shown, the data collection module and the data conversion module can be used to place real-time conversation messages and target offline messages into the first pending message queue. For the first pending message queue topic, the data collection module that collects real-time conversation messages can serve as producer 1 (i.e., producer 1) of topic 1, and the data conversion module that converts the target offline file into a preset message can serve as producer 2 (i.e., producer 2) of the topic.

[0074] Before performing duplicate checking on the messages in the first queue of pending messages based on the preset distributed data duplicate checking service, the messages in the first queue of pending messages can be pre-processed. In this pre-processing step, the messages in the first queue of pending messages can be pre-checked for duplicates through message subscription (Subscription1). The message subscription can be integrated into one subscription according to technical support needs, or it can be partitioned according to location information, business type, etc., and then consumers in different partitions (i.e., distributed data pre-processing services) can pre-check for duplicates on the messages in the first queue of pending messages (such as pre-checking for duplicates based on the first-level duplicate checking memory library) to speed up data processing efficiency.

[0075] After preprocessing the messages in the first queue, the preprocessed messages can be placed back into the queue. For topic 1 in the first queue, each distributed data preprocessing service acts as a consumer of topic 1. For topic 2 in the first queue, each distributed data preprocessing service acts as a producer of topic 2.

[0076] During data processing, the failover subscription model can be used to ensure the stability of the system during data processing. For example, Figure 5 As shown, the failover subscription mode can be a main consumer (i.e. consumer1) and a preset number of backup consumers (such as consumer2 and consumer3) to process messages. Once the main consumer is disconnected, a transferable consumer (such as consumer3) can be selected from the backup consumers as the main consumer to continue processing messages, which can improve the reliability of streaming processing.

[0077] In addition, based on the target business processing service, when processing messages in the target business processing queue, if the communication business is a business with strict timing requirements, the corresponding target business processing queue can be a streaming queue, that is, the earlier the time, the queue is entered first, and the later the time, and the message queue can only be processed in the failover subscription mode in streaming processing.

[0078] For exclusive subscriptions (i.e., there can only be one recipient for the same message), if the communication business has strict requirements on timing, then if a shared subscription model is used for batch processing, it will cause the problem of disordered execution of service processing instructions. Therefore, the messages can be processed in an exclusive mode of streaming. According to the business rules, the messages can be sorted into different service activation queues (equivalent to independent channels), and the messages can also be processed in a streaming mode, that is, the messages are distributed to multiple preset distributed business processing services. On the message producer side, the messages can be processed in different channels based on the business characteristics of the communication business. For example, for account activation type messages, they can be diverted according to the account to ensure that the same account activation messages enter the same stream channel, that is, distributed to the same distributed data processing service.

[0079] In this way, based on the above-mentioned data processing system built on Pulsar, it is possible to achieve the connection between distributed transaction processing and cloud data processing framework, realize duplicate checking of messages, support message query operations in multiple scenarios (such as maintenance scenarios and monitoring scenarios), support data accumulation and time-sharing output in multiple environments such as message generation, accounting, and settlement, and realize full-link tracking of the message flow process. It can provide operators with business data management functions and realize unified management of the life cycle of global business data.

[0080] In addition, the data processing system built on Pulsar can support the separation of storage and services, and the messaging system is a cloud-native framework design that can be well connected with the application layer of the cloud-based business processing system, achieving ultra-high throughput business processing capabilities.

[0081] The retention period of different messages in the data processing system built on Pulsar is configurable. Messages that need to be persisted can be copied to the database in real time or periodically through the data management function for use by relevant staff in business scenarios such as query, statistics, and monitoring.

[0082] In addition, the data processing system built on Pulsar can support multi-center off-site replication. This feature is also compatible with the multi-center framework supported by the cloud data processing system. To ensure robustness and disaster recovery capabilities, the cloud data processing system needs to be deployed in a multi-center form. The centers back up each other and can switch online. In this way, smooth business migration can be achieved in failure scenarios, ensuring data integrity and efficient data replication between centers.

[0083] An embodiment of the present invention provides a data processing method that, based on a preset data interface, collects real-time conversation messages and target offline files generated during the processing of communication services. The target offline files are used to settle the communication services during a preset settlement period. The target offline files are processed into preset messages, converted into target offline messages, and the real-time conversation messages and the target offline messages are placed in a first message queue to be processed. Based on a preset distributed data processing service, the messages in the first message queue to be processed are processed and placed in a second message queue to be processed, so that the communication services are processed based on the messages in the second message queue. In this way, processing real-time conversation messages through a distributed data processing service can avoid the low data processing efficiency problem of shared storage-based data processing methods due to the need to frequently land real-time conversation messages. In addition, based on this method, real-time conversation messages and target offline files can be processed simultaneously, avoiding the low data processing efficiency problem caused by using two methods (i.e., shared storage-based data processing methods and message system-based data processing methods) to process different data (i.e., real-time conversation messages and target offline files) separately, thereby improving data processing efficiency when processing communication services.

[0084] Example 3

[0085] The above is a data processing method provided by an embodiment of the present invention. Based on the same idea, an embodiment of the present invention also provides a data processing device, such as Figure 6 shown.

[0086] The data processing device includes: a data acquisition module 601, a first processing module 602 and a second processing module 606, wherein:

[0087] The data acquisition module 601 is used to collect real-time conversation messages and target offline files generated during the communication service processing based on a preset data interface. The target offline files are used to settle the communication service in a preset settlement period.

[0088] A first processing module 602 is configured to perform preset message conversion on the target offline file, convert the target offline file into a target offline message, and place the real-time conversation message and the target offline message into a first message queue to be processed;

[0089] The second processing module 603 is used to process the messages in the first message queue to be processed based on the preset distributed data processing service, and put the processed messages into the second message queue to be processed, so as to process the communication service based on the messages in the second message queue to be processed.

[0090] In this embodiment of the present invention, the first processing module 602 is configured to:

[0091] Based on the preset distributed data duplicate checking service, the messages in the first message queue to be processed are checked for duplicates, and the messages after the duplicate checking are placed in the second message queue to be processed.

[0092] In this embodiment of the present invention, the first processing module 602 is configured to:

[0093] Sending the messages in the first to-be-processed message queue to the plurality of distributed data duplicate checking processing services respectively;

[0094] Each of the distributed data duplicate checking and processing services performs duplicate checking on the messages received in the first message queue to be processed based on a preset full database, and places the duplicate checked messages into the second message queue to be processed.

[0095] In this embodiment of the present invention, the first processing module 602 is configured to:

[0096] Put the message after the duplicate checking process into the third queue of pending messages;

[0097] Based on the preset distributed data accounting service, the messages in the third message queue to be processed are accounted for, and the accounted messages are placed in the second message queue to be processed.

[0098] In an embodiment of the present invention, the device further includes:

[0099] a third processing module, configured to place the messages in the second to-be-processed message queue into a target service processing queue corresponding to the service processing type of the communication service;

[0100] The fourth processing module is configured to obtain a target business processing service corresponding to the target business processing queue, and process the messages in the target business processing queue based on the target business processing service.

[0101] In this embodiment of the present invention, the fourth processing module is configured to:

[0102] Based on the target business processing service, generating a processing task corresponding to each message in the target business processing queue;

[0103] Based on the processing time of the processing task, the processing task is sequentially placed into the corresponding business processing queue, so that the processing tasks contained in the business processing queue are processed based on the processing time of the processing task.

[0104] In an embodiment of the present invention, the device further includes:

[0105] The fifth processing module is used to put the duplicate messages in the first to-be-processed message queue into an abnormal message queue, and process the messages in the abnormal message queue based on a preset abnormal message processing service.

[0106] An embodiment of the present invention provides a data processing device that, based on a preset data interface, collects real-time conversation messages and target offline files generated during the processing of communication services. The target offline files are used to settle the communication services during a preset settlement period. The target offline files are processed into preset messages, converted into target offline messages, and the real-time conversation messages and the target offline messages are placed in a first message queue to be processed. Based on a preset distributed data processing service, the messages in the first message queue to be processed are processed and placed in a second message queue to be processed, so that the communication services are processed based on the messages in the second message queue. In this way, processing real-time conversation messages through a distributed data processing service can avoid the low data processing efficiency problem of shared storage-based data processing methods due to the need to frequently land real-time conversation messages. In addition, based on this method, real-time conversation messages and target offline files can be processed simultaneously, avoiding the low data processing efficiency problem caused by using two methods (i.e., shared storage-based data processing methods and message system-based data processing methods) to process different data (i.e., real-time conversation messages and target offline files) separately, thereby improving data processing efficiency when processing communication services.

[0107] Example 4

[0108] Figure 7 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention is provided below.

[0109] The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, a processor 710, and a power supply 711. It will be understood by those skilled in the art that Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. In the embodiments of the present invention, the electronic device includes but is not limited to a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle terminal, a wearable electronic device, and a pedometer.

[0110] Among them, the processor 710 is used to: based on a preset data interface, collect real-time conversation messages and target offline files generated during the communication service processing, and the target offline files are used to settle the communication service in a preset settlement period; perform preset message conversion on the target offline files, convert the target offline files into target offline messages, and place the real-time conversation messages and the target offline messages into a first message queue to be processed; based on a preset distributed data processing service, process the messages in the first message queue to be processed, and place the processed messages into a second message queue to be processed, so as to process the communication service based on the messages in the second message queue to be processed.

[0111] In addition, the processor 710 is further configured to perform duplicate checking on the messages in the first message queue to be processed based on a preset distributed data duplicate checking service, and place the duplicate checked messages into the second message queue to be processed.

[0112] In addition, the processor 710 is also used to: send the messages in the first message queue to be processed to multiple distributed data duplication checking processing services respectively; each distributed data duplication checking processing service performs duplication checking on the messages received in the first message queue to be processed based on the preset full database, and puts the messages after duplication checking into the second message queue to be processed.

[0113] In addition, the processor 710 is also used to: place the message after the duplicate checking process into the third message queue to be processed; based on the preset distributed data accounting service, perform accounting processing on the messages in the third message queue to be processed, and place the message after the accounting processing into the second message queue to be processed.

[0114] In addition, the processor 710 is also used to: place the messages in the second message queue to be processed into a target business processing queue corresponding to the business processing type of the communication business; obtain a target business processing service corresponding to the target business processing queue, and process the messages in the target business processing queue based on the target business processing service.

[0115] In addition, the processor 710 is also used to: generate a processing task corresponding to each message in the target business processing queue based on the target business processing service; and place the processing tasks into the corresponding business processing queue in sequence based on the processing time of the processing tasks, so as to process the processing tasks contained in the business processing queue based on the processing time of the processing tasks.

[0116] In addition, the processor 710 is further configured to: place duplicate messages in the first to-be-processed message queue into an abnormal message queue, and process the messages in the abnormal message queue based on a preset abnormal message processing service.

[0117] An embodiment of the present invention provides an electronic device that, based on a preset data interface, collects real-time conversation messages and target offline files generated during the processing of communication services. The target offline files are used to settle the communication services during a preset settlement period. The target offline files are processed into preset messages, converted into target offline messages, and the real-time conversation messages and the target offline messages are placed in a first message queue to be processed. Based on a preset distributed data processing service, the messages in the first message queue to be processed are processed and placed in a second message queue to be processed, so that the communication services are processed based on the messages in the second message queue to be processed. In this way, by processing real-time conversation messages through a distributed data processing service, the problem of low data processing efficiency caused by the need to frequently land real-time conversation messages in a data processing method based on shared storage can be avoided. In addition, based on this method, real-time conversation messages and target offline files can be processed simultaneously, avoiding the problem of low data processing efficiency caused by using two methods (i.e., a data processing method based on shared storage and a data processing method based on a message system) to process different data (i.e., real-time conversation messages and target offline files) separately, thereby improving data processing efficiency when processing communication services.

[0118] It should be understood that in this embodiment of the present invention, the RF unit 701 can be used to receive and transmit signals during information transmission or calls. Specifically, it receives downlink data from the base station and transmits it to the processor 710 for processing; in addition, it transmits uplink data to the base station. Typically, the RF unit 701 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and the like. Furthermore, the RF unit 701 can communicate with the network and other electronic devices via a wireless communication system.

[0119] The electronic device provides users with wireless broadband Internet access through the network module 702, such as helping users to send and receive emails, browse web pages, and access streaming media.

[0120] The audio output unit 703 can convert audio data received by the RF unit 701 or the network module 702 or stored in the memory 709 into an audio signal and output it as sound. In addition, the audio output unit 703 can also provide audio output related to a specific function performed by the electronic device 700 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 703 includes a speaker, a buzzer, a receiver, etc.

[0121] The input unit 704 is used to receive audio or video signals. The input unit 704 may include a graphics processing unit (GPU) 7041 and a microphone 7042. The graphics processor 7041 processes image data of a still picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frames can be displayed on the display unit 706. The image frames processed by the graphics processor 7041 can be stored in the memory 709 (or other storage medium) or transmitted via the radio frequency unit 701 or the network module 702. The microphone 7042 can receive sound and can process such sound into audio data. The processed audio data can be converted into a format that can be sent to a mobile communication base station via the radio frequency unit 701 in the case of a telephone call mode.

[0122] The electronic device 700 also includes at least one sensor 705, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 7061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 7061 and / or the backlight when the electronic device 700 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 705 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be repeated here.

[0123] The display unit 706 is used to display information input by the user or information provided to the user. The display unit 706 may include a display panel 7061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0124] The user input unit 707 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the electronic device. Specifically, the user input unit 707 includes a touch panel 7071 and other input electronic devices 7072. The touch panel 7071, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by the user using any suitable object or accessory such as a finger, stylus, etc. on or near the touch panel 7071). The touch panel 7071 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch direction and detects the signal caused by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 710, which receives and executes the command sent by the processor 710. In addition, the touch panel 7071 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 7071, the user input unit 707 may also include other input electronic devices 7072. Specifically, other input electronic devices 7072 may include but are not limited to a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0125] Furthermore, the touch panel 7071 may be overlaid on the display panel 7061. When the touch panel 7071 detects a touch operation on or near it, it transmits the information to the processor 710 to determine the type of touch event. Subsequently, the processor 710 provides corresponding visual output on the display panel 7061 according to the type of touch event. Figure 7 In the figure, the touch panel 7071 and the display panel 7061 are two independent components to realize the input and output functions of the electronic device. However, in some embodiments, the touch panel 7071 and the display panel 7061 can be integrated to realize the input and output functions of the electronic device, which is not limited here.

[0126] The interface unit 708 is an interface for connecting external devices to the electronic device 700. For example, the external devices may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 708 may be used to receive input (e.g., data information, power, etc.) from the external device and transmit the received input to one or more elements within the electronic device 700, or may be used to transmit data between the electronic device 700 and the external device.

[0127] Memory 709 can be used to store software programs and various data. Memory 709 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). Furthermore, memory 709 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0128] The processor 710 is the control center of the electronic device. It connects the various components of the electronic device using various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 709 and accessing data stored in the memory 709, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 710 may include one or more processing units; preferably, the processor 710 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 710.

[0129] The electronic device 700 may also include a power supply 711 (such as a battery) to supply power to each component. Preferably, the power supply 711 may be logically connected to the processor 710 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0130] Preferably, an embodiment of the present invention further provides an electronic device, comprising a processor 710, a memory 709, and a computer program stored in the memory 709 and executable on the processor 710. When the computer program is executed by the processor 710, the various processes of the above-mentioned data processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0131] Example 5

[0132] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the various processes of the above-described data processing method embodiment and achieves the same technical effects. To avoid repetition, the details are not described here. The computer-readable storage medium may be, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0133] An embodiment of the present invention provides a computer-readable storage medium that, based on a preset data interface, collects real-time conversation messages and target offline files generated during the processing of communication services. The target offline files are used to settle the communication services during a preset settlement period. The target offline files are processed into preset messages, converted into target offline messages, and the real-time conversation messages and the target offline messages are placed in a first message queue to be processed. Based on a preset distributed data processing service, the messages in the first message queue to be processed are processed and placed in a second message queue to be processed, so that the communication services are processed based on the messages in the second message queue. In this way, processing real-time conversation messages through a distributed data processing service can avoid the low data processing efficiency problem of shared storage-based data processing methods due to the need to frequently land real-time conversation messages. In addition, based on this method, real-time conversation messages and target offline files can be processed simultaneously, avoiding the low data processing efficiency problem caused by using two methods (i.e., shared storage-based data processing methods and message system-based data processing methods) to process different data (i.e., real-time conversation messages and target offline files) separately, thereby improving data processing efficiency when processing communication services.

[0134] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] The present invention is described with reference to flowcharts and / or block diagrams of methods, electronic devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing electronic device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing electronic device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing electronic device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing electronic device so that a series of operating steps are executed on the computer or other programmable electronic device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable electronic device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0138] In a typical configuration, a computing electronic device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0139] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0140] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage electronic devices or any other non-transmission media that can be used to store information that can be accessed by computing electronic devices. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0141] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or electronic device that includes a list of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or electronic device. In the absence of further limitations, an element defined by the phrase "comprises a..." does not preclude the presence of additional identical elements in the process, method, commodity, or electronic device that includes the element.

[0142] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] The foregoing is merely an embodiment of the present invention and is not intended to limit the present invention. It will be apparent to those skilled in the art that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention are intended to be included within the scope of the claims of the present invention.

Claims

1. A data processing method, characterized in that: The method comprises: Based on a preset data interface, real-time conversation messages and target offline files generated during the communication service processing are collected, and the target offline files are used to settle the communication service in a preset settlement period; the preset data interface is any interface that can collect the real-time conversation messages and the target offline files from the network element side device; Performing preset message conversion processing on the target offline file, converting the target offline file into a target offline message, and placing the real-time conversation message and the target offline message into a first to-be-processed message queue; Based on a preset distributed data processing service, processing the messages in the first message queue to be processed, and placing the processed messages into a second message queue to be processed, so as to process the communication service based on the messages in the second message queue to be processed; The method of processing the messages in the first message queue to be processed based on the preset distributed data processing service and placing the processed messages into the second message queue to be processed includes: Based on the preset distributed data duplicate checking service, the messages in the first message queue to be processed are checked for duplicates, and the messages after the duplicate checking are placed in the second message queue to be processed.

2. The method according to claim 1, characterized in that The method of performing duplicate checking on the messages in the first message queue to be processed based on the preset distributed data duplicate checking service and placing the duplicate checked messages into the second message queue to be processed includes: Sending the messages in the first to-be-processed message queue to the plurality of distributed data duplicate checking processing services respectively; Each of the distributed data duplicate checking and processing services performs duplicate checking on the messages received in the first message queue to be processed based on a preset full database, and places the duplicate checked messages into the second message queue to be processed.

3. The method according to claim 2, characterized in that The step of placing the message after duplicate checking into the second queue of messages to be processed includes: Put the message after the duplicate checking process into the third queue of pending messages; Based on the preset distributed data accounting service, the messages in the third message queue to be processed are accounted for, and the accounted messages are placed in the second message queue to be processed.

4. The method according to claim 3, characterized in that After the message after accounting processing is placed into the second queue of messages to be processed, the method further includes: placing the messages in the second to-be-processed message queue into a target service processing queue corresponding to the service processing type of the communication service; A target business processing service corresponding to the target business processing queue is acquired, and based on the target business processing service, the message in the target business processing queue is processed.

5. The method according to claim 4, characterized in that The processing of the message in the target business processing queue based on the target business processing service includes: Based on the target business processing service, generating a processing task corresponding to each message in the target business processing queue; Based on the processing time of the processing task, the processing task is sequentially placed into the corresponding business processing queue, so that the processing tasks contained in the business processing queue are processed based on the processing time of the processing task.

6. The method according to claim 1, characterized in that The method further comprises: The duplicate messages in the first to-be-processed message queue are placed into an abnormal message queue, and the messages in the abnormal message queue are processed based on a preset abnormal message processing service.

7. A data processing device, characterized in that: The device comprises: A data acquisition module is configured to collect, based on a preset data interface, real-time conversation messages and target offline files generated during the communication service processing process, the target offline files being used for settlement processing of the communication service in a preset settlement period; the preset data interface is any interface capable of collecting the real-time conversation messages and the target offline files from a network element-side device; a first processing module configured to perform preset message conversion on the target offline file, convert the target offline file into a target offline message, and place the real-time conversation message and the target offline message into a first message queue to be processed; a second processing module, configured to process the messages in the first message queue to be processed based on a preset distributed data processing service, and place the processed messages into a second message queue to be processed, so as to process the communication service based on the messages in the second message queue to be processed; In the second processing module, based on a preset distributed data processing service, the messages in the first message queue to be processed are processed, and the processed messages are placed in a second message queue to be processed, including: Based on the preset distributed data duplicate checking service, the messages in the first message queue to be processed are checked for duplicates, and the messages after the duplicate checking are placed in the second message queue to be processed.

8. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the data processing method according to any one of claims 1 to 6 when executed by the processor.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 6 are implemented.

Images