A data processing method

Through the collaborative work between the interactive server and the data processing end, the account number and password are parsed by the specified file, the task type is determined and processed, and the problems of low data source access efficiency and high management cost are solved, and low-cost and efficient data processing is achieved.

CN114036368BActive Publication Date: 2025-07-25BEIJING MININGLAMP SOFTWARE SYST CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202110964991.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-20
Publication Date
2025-07-25
Estimated Expiration
2041-08-20

AI Technical Summary

Technical Problem

In the prior art, data source access efficiency is low, and data processing platform maintenance and management costs are high.

Method used

The interactive server sends the specified file to the data processing end, parses out the account and password, establishes a connection, determines the task type and processes it according to the data type, and reduces the system architecture construction requirements of the data processing end.

Benefits of technology

It reduces the threshold for the data processing terminal to obtain pending data from the data source, reduces the cost of system architecture construction, and improves the efficiency and flexibility of data processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114036368B_ABST
    Figure CN114036368B_ABST
Patent Text Reader

Abstract

The present application relates to a data processing method. In this method, when the data processing end receives a specified file from the interactive server, it parses the account number and password for accessing the data source from the specified file, where the data source stores the data to be processed; uses the account number and password to establish a connection with the data source; determines the type of task corresponding to the data to be processed according to the type of the data to be processed in the data source as the specified type; obtains the task corresponding to the specified type according to the data to be processed; and processes the task to obtain a processing result. In this method, the interactive server sends the account number and password required to obtain the data to be processed to the data processing end through the specified file, so that the data processing end does not need to rebuild the system architecture for accessing the data source, and can obtain the data to be processed from the data source, reducing the threshold for the data processing end to obtain the data to be processed from the data source, which is beneficial to reducing the cost of data processing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the technical field of data processing, and in particular, to a data processing method. Background Art

[0002] With the continuous maturity of mobile Internet technology, a large amount of data floods people's production and life. There are various ways to apply data, and a data processing platform that provides data processing services for enterprises and groups (hereinafter collectively referred to as users) has emerged as the times require.

[0003] Generally, different users have different requirements for the processing results of data, and the data processing platform needs to provide frameworks and interfaces adapted to the needs of different users. When the number of connected users is large, the workload of maintaining and managing the frameworks and interfaces by the data processing platform is also large. If a new data source is accessed, new frameworks and interfaces need to be developed for the new user, increasing the cost of user access. Summary of the Invention

[0004] This application provides a data processing method to solve the problems of low data source access efficiency and high maintenance and management costs of the data processing platform in the prior art.

[0005] In a first aspect, this application provides a first data processing method. The method is applied to a data processing end, and the method includes: when receiving a specified file from an interaction server, parsing out an account and a password for accessing a data source from the specified file, where the data source stores data to be processed; using the account and the password to establish a connection with the data source; determining, according to the type of the data to be processed in the data source, the type of a task corresponding to the data to be processed as a specified type, where the type of the task includes at least one of the following: Simple type, Stream type, Script type; obtaining, according to the data to be processed, a task corresponding to the specified type; and processing the task to obtain a processing result.

[0006] In an optional embodiment of this specification, the specified file includes a jar package.

[0007] In an optional embodiment of this specification, before parsing out an account and a password for accessing a data source from the specified file when receiving a specified file from an interaction server, the method further includes: registering first interface information of the data processing end to a service discovery platform, where the first interface information is used to implement a communication connection with the interaction server.

[0008] In an optional embodiment of the present specification, the data processing end is based on a task processing system, and the task processing system includes a registration center and a plurality of nodes. Among them, processing the task to obtain a processing result includes: the master node among the plurality of nodes performs sharding processing on the task to obtain a plurality of shards; the master node persists the shard markers of the plurality of shards to the registration center; each node among the plurality of nodes grabs the shard corresponding to the shard marker in the registration center as the target shard for each shard marker in the registration center; if the status of the target shard is in running, the processing method corresponding to the type of the task to which the target shard belongs is used to process the target shard to obtain the sub-processing result corresponding to the target shard; the sub-processing results corresponding to each shard are summarized to obtain the processing result. The sharding rule is to process according to the CPU and memory occupancy of the current node. When the CPU and memory occupancy ratios are very high, tasks will not be run on this node.

[0009] In an optional embodiment of the present specification, after detecting a specified file and parsing out the account and password for obtaining the data to be processed from the specified file, the method further includes: if new account and password sent by the interaction server are received, the account and password parsed out from the specified file are updated according to the new account and password.

[0010] In addition, if the data processing end cannot establish a connection with the data source using the account and password parsed out from the specified file, the data processing end generates a specified file error prompt message and sends it to the interaction server, so that the user can regenerate the specified file according to the specified file error prompt message displayed by the interaction server.

[0011] In an optional embodiment of the present specification, before obtaining the specified file, the method further includes: sending a data source information template to the interaction server, so that the interaction server generates a front-end page displaying the data source information template, where the data source information template includes a plurality of fields for adding an account and a password.

[0012] In a second aspect, the present application provides a second data processing method, which is applied to an interaction server, and the method includes:

[0013] When the specified file is obtained, send the specified file to the data processing end, so that the data processing end parses the account number and password for obtaining the data to be processed from the specified file, and then according to the account number and password, obtain the data to be processed from the data source corresponding to the account number and password, and determine the type of the task corresponding to the data to be processed according to the type of the obtained data to be processed, as the specified type, the type of the task includes at least one of the following: Simple type, Stream type, Script type; obtain the task corresponding to the specified type according to the data to be processed; process the task to obtain a processing result.

[0014] In an optional embodiment of the present specification, before sending the specified file to the data processing end when the specified file is obtained, the method further includes: generating a front-end page and obtaining the specified file uploaded through the front-end page; and / or registering the second interface information of the interactive server to the service discovery platform, where the second interface information is used to implement a communication connection with the data processing end.

[0015] In an optional embodiment of the present specification, generating a front-end page includes: receiving the data source information template of the data processing end, where the data source information template includes a plurality of fields for adding an account number and a password; generating a front-end page, where the data source information template is displayed in the front-end page.

[0016] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete mutual communication through the communication bus;

[0017] The memory is used to store a computer program;

[0018] The processor is used to implement the steps of any data processing method in the first aspect or the steps of any data processing method in the second aspect when executing the program stored in the memory.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and the computer program implements the steps of any data processing method in the first aspect or the steps of any data processing method in the second aspect when being executed by a processor.

[0020] The above technical solutions provided by the embodiments of the present application have the following advantages compared with the prior art:

[0021] The method provided by the embodiments of this application is applied to the field of data capabilities and can be applied to data transmission. In the data processing method in this specification, the interactive server sends the account number and password required to obtain the data to be processed to the data processing end through a specified file. Then, the data processing end does not need to rebuild the system architecture for accessing the data source, that is, it can obtain the data to be processed from the data source, reducing the threshold for the data processing end to obtain the data to be processed from the data source, which is beneficial to reducing the cost of data processing. Description of the Drawings

[0022] The drawings here are incorporated into the specification and form a part of this specification, showing the embodiments consistent with the present invention and used together with the specification to explain the principles of the present invention.

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0024] Figure 1 It is a schematic diagram of the scenario involved in the data processing process provided by the embodiments of this application;

[0025] Figure 2 It is a schematic flowchart of the data processing process provided by the embodiments of this application;

[0026] Figure 3 It is a schematic diagram of the process of the processing task in the data processing process provided by the embodiments of this application;

[0027] Figure 4 Corresponding to Figure 2 It is a schematic diagram of the first data processing device corresponding to some steps in the method process;

[0028] Figure 5 Corresponding to Figure 2 It is a schematic diagram of the second data processing device corresponding to some steps in the method process;

[0029] Figure 6 It is a schematic diagram of the structure of an electronic device provided by the embodiments of this application. Detailed Embodiments

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0031] The data processing method in this specification is based on a data processing terminal and an interaction server. The data processing terminal is used to process the data to be processed, and the interaction server is used to implement the access of the data source to the data processing terminal. The scenarios involved are, for example, as Figure 1 shown.

[0032] such as Figure 2 shown, the data processing method in this specification includes the following steps:

[0033] S200: When the interaction server obtains a specified file, it sends the specified file to the data processing terminal.

[0034] The interaction server in this specification is used to interact with the user to obtain a specified file based on the interaction with the user. The interaction server can provide a Web service.

[0035] In an optional embodiment of this specification, the interaction server generates a front-end page, displays the front-end page to the user, and guides the user to upload a specified file through the front-end page, so as to obtain the specified file uploaded by the user from the front-end page.

[0036] S202: When the data processing terminal receives the specified file from the interaction server, it parses the account and password for obtaining the data to be processed from the specified file.

[0037] The data source stores the data to be processed, and the specified file represents the way to access the data source. In this specification, the specified file contains data for establishing a connection with the data source. Exemplarily, the specified file can be a jar package.

[0038] S204: The data processing terminal uses the account and password to establish a connection with the data source.

[0039] In an optional embodiment of this specification, the data processing process in this specification is applied to process the data generated and / or displayed by an application client (for example, application). In this embodiment, the application client is the data source, and the data generated and / or displayed by the application client is the data to be processed.

[0040] In this embodiment, to avoid the data processing terminal from specifically building an architecture and maintaining interfaces for the application client, the specified file may contain an executable file of the application client. Then, the data processing terminal parses the executable file of the application client from the specified file, runs the executable file, and uses the account and password parsed from the specified file to log in to the application client corresponding to the executable file to establish a connection with the application client. Then, it obtains the data to be processed from the application client.

[0041] Exemplarily, when the application client is an instant messaging client, the instant messaging client is the data source, and the interaction data (e.g., chat records) generated by the interaction process implemented through the instant messaging client is the data to be processed.

[0042] In addition, the application client can also be a community client, which is the data source, and the data published by the user through the community client is the data to be processed. Among them, the "community" can be a video sharing community, a commodity trading community, a forum, etc. The data published by the user through the community client can be videos, trading information, news, etc. published by the user.

[0043] This specification does not specifically limit the timing of the data processing end to establish a connection with the data source. In an optional embodiment of this specification, the data processing end can periodically establish a connection with the data source to periodically obtain the data to be processed. This embodiment is applicable to the situation where the data in the data source is not updated quickly (i.e., the amount of newly added data is small).

[0044] In another optional embodiment of this specification, the data processing end maintains a connection with the data source to obtain the data to be processed from the data source in real time. This embodiment is applicable to the situation where the data in the data source is updated relatively quickly.

[0045] S206: The data processing end determines the type of the task corresponding to the data to be processed in the data source as the specified type according to the type of the data to be processed.

[0046] The types of tasks in this specification may include at least one of the following: Simple type, Stream type, Script type. Exemplarily, the Script type means a script type job, supporting all types of scripts such as shell, python, perl, etc.

[0047] This step determines the type of the task, that is, determines the processing method of the task.

[0048] S208: The data processing end obtains the task corresponding to the specified type according to the data to be processed.

[0049] In an optional embodiment of this specification, the data processing end is based on a task processing system, and the task processing system may include a scheduling center (elastic-job), and this step can be executed by the scheduling center.

[0050] S210: The data processing end processes the task to obtain a processing result.

[0051] The tasks in this specification can be processed by a task processing system. Continuing from the above embodiments, the task processing system may further include a registration center and several nodes, where any one of the nodes has a certain degree of task processing ability.

[0052] Optionally, the master node among the several nodes performs sharding processing on the task to obtain several shards. The master node persists the shard markers of the several shards to the registration center. Each of the several nodes fetches the shard corresponding to each shard marker in the registration center as the target shard.

[0053] If the status of the target shard is in running, then the target shard is processed using the processing method corresponding to the type of the task to which the target shard belongs to obtain the sub-processing result corresponding to the target shard. Then, in the registration center, the status of the target shard is adjusted to end. After that, the sub-processing results corresponding to each shard are aggregated to obtain the processing result. Each node can be managed by zookeeper.

[0054] It can be seen that through the data processing process in this specification, the interaction server sends the account and password required to obtain the data to be processed to the data processing end through a specified file. Then, the data processing end does not need to rebuild the system architecture for accessing the data source, that is, it can obtain the data to be processed from the data source, reducing the threshold for the data processing end to obtain the data to be processed from the data source, which is beneficial to reducing the cost of data processing. It also enables the data processing end in this specification to perform data processing in parallel for multiple data sources without causing chaos due to excessive access to data sources.

[0055] In an optional embodiment of this specification, before step S200, to establish a communication connection between the data processing end and the interaction server. The data processing end registers its first interface information to the service discovery platform. In this specification, the first interface information is used to establish a communication connection with the interaction server. When the interaction server obtains the first interface information, it can establish a connection with the data processing end through the first interface information. The service discovery platform (for example, Nacos) is used to provide service discovery, based on DNS and / or RPC, and can serve as the registration center, dynamic configuration service, dynamic DNS service, etc. of springcloud.

[0056] The interaction server registers its second interface information to the service discovery platform. The second interface information is used to establish a communication connection with the data processing end. After the interaction server obtains the first interface information from the service discovery platform, it can establish a communication connection with the data processing end to prepare for subsequent data processing steps.

[0057] It can be seen that both the data processing end and the interaction server end register the interface information required for establishing a communication connection to the service discovery platform. Then, either the data processing end or the interaction server end can establish a communication connection with the other end through the discovery service provided by the service discovery platform, making the method in this specification highly flexible. If a new interaction server end appears, there is no need for a complex communication connection establishment process, and one or more interaction server ends can be efficiently and conveniently connected to the data processing end.

[0058] In an actual data processing scenario, to prevent criminals from stealing data from the data source and / or injecting illegal data into data processing by taking advantage of the convenient way of accessing the data source in the data processing process described in this specification, it is possible that the data source changes its account and / or password. In an optional embodiment of this specification, if the data processing end receives new account and password sent by the interaction server end, it updates the account and password parsed from the specified file according to the new account and password. Obtaining the data to be processed from the data source corresponding to the account and password includes: establishing a connection with the data source corresponding to the updated account and password using the updated account and password. Obtaining the data to be processed from the data source.

[0059] Optionally, the data processing end can also generate a risk prompt message and send it to the interaction server end under certain conditions (for example, detecting that the duration of the account and / or password of a certain data source since the last update is greater than a preset duration threshold, which means meeting certain conditions), so that the interaction server end displays the risk prompt message. The risk prompt message is used to prompt the user to update the account and / or password.

[0060] In an optional embodiment of this specification, to guide the user to perform account and / or password update and upload the specified file, the data processing end sends the data source information template to the interaction server end, so that the interaction server end generates a front-end page displaying the data source information template, where the data source information template includes several fields for adding the account and password. Then, the user can fill in at least part of the fields in the data source information template through the interaction interface provided by the interaction server end.

[0061] Now, in an embodiment, how any node in each node performs task processing will be described.

[0062] As Figure 3As shown in the figure. The node detects whether there is a task to be sharded. If the detection result is no, it waits for the first specified duration (a preset duration, which can be determined according to actual requirements) and then performs the detection of the task to be sharded again. If the detection result is yes, it determines whether the current is in the master node election. If the current is in the master node election, it sleeps for the second specified duration (for example, 100 milliseconds) and determines whether this node itself is the master node. If the current is not in the master node election, it determines whether this node itself is the master node.

[0063] If this node itself is the master node, it shards the task to be sharded and persists the shard markers of the obtained shards to the registration center. Then, for each shard marker in the registration center, it grabs the shard corresponding to the shard marker as the target shard.

[0064] If this node itself is not the master node, for each shard marker in the registration center, it grabs the shard corresponding to the shard marker as the target shard.

[0065] It determines the status of the target shard. If the status of the target shard is in running, it processes the target shard using the processing method corresponding to the type of the task to which the target shard belongs. Specifically, if the status of the target shard is in running, it determines the type of the task to which the target shard belongs.

[0066] If the type of the task to which the target shard belongs is the simple type, it executes the job running interface.

[0067] If the type of the task to which the target shard belongs is dataflow, it executes the data acquisition interface to acquire the data to be processed from the data source. It determines whether the data corresponding to the target shard is acquired from the data source. If the data corresponding to the target shard is not acquired from the data source, it adjusts the status of the target shard to ended in the registration center. If the data corresponding to the target shard is acquired from the data source, it splits the acquired data, distributes the obtained sub-data to each thread. Each thread processes the sub-data to obtain sub-processing results. It sends the sub-processing results to kafka, and the downstream system processes each sub-processing result to obtain the processing result. Then, it destroys the corresponding classloader.

[0068] In an optional embodiment of the present specification, after the data processing end generates the processing result, it sends the processing result to the interactive server, so that the interactive server receives the processing result and then displays the processing result to the user.

[0069] Based on the same idea, the present specification further provides the first data processing device, which is applied to the data processing end, as Figure 4As shown in the figure, the first data processing device includes one or more of the following modules:

[0070] A specified file receiving module 400, configured to: when receiving a specified file from an interactive server, parse an account and a password for accessing a data source from the specified file, where the data source stores data to be processed.

[0071] A connection establishment module 402, configured to: establish a connection with the data source using the account and the password.

[0072] A task type determination module 404, configured to: determine the type of the task corresponding to the data to be processed in the data source as a specified type according to the type of the data to be processed in the data source, where the type of the task includes at least one of the following: Simple type, Stream type, Script type.

[0073] A task generation module 406, configured to: obtain a task corresponding to the specified type according to the data to be processed.

[0074] A processing result generation module 408, configured to: process the task to obtain a processing result.

[0075] In an optional embodiment of this specification, the specified file includes a jar package.

[0076] In an optional embodiment of this specification, the first data processing device further includes a first registration module, configured to register the first interface information of the data processing end to a service discovery platform, where the first interface information is used to implement a communication connection with the interactive server.

[0077] In an optional embodiment of this specification, the processing result generation module 408 is specifically configured to: the master node among the several nodes performs sharding processing on the task to obtain several shards;

[0078] The master node persists the shard markers of the several shards to the registration center;

[0079] The several nodes

[0080] Each node among them grabs the shard corresponding to the shard marker in the registration center as a target shard for each shard marker in the registration center;

[0081] If the status of the target shard is in running, the target shard is processed using the processing method corresponding to the type of the task to which the target shard belongs to obtain a sub-processing result corresponding to the target shard, and in the registration center, the status of the target shard is adjusted to end;

[0082] Summarize the sub - processing results corresponding to each shard to obtain the processing result.

[0083] In an optional embodiment of this specification, the first data - processing device further includes an update module, configured to: if new account and password sent by the interactive server are received, update the account and password parsed from the specified file according to the new account and password.

[0084] In an optional embodiment of this specification, the first data - processing device further includes a data - source information template sender, configured to send the data - source information template to the interactive server, so that the interactive server generates a front - end page displaying the data - source information template, where the data - source information template includes several fields for adding an account and a password.

[0085] Based on the same idea, this specification further provides a second data - processing device, as Figure 5 shown. The second data - processing device includes one or more of the following modules:

[0086] A specified - file sending module 500, configured to, when the specified file is obtained, send the specified file to the data - processing end, so that the data - processing end parses the account and password for obtaining the data to be processed from the specified file, and then according to the account and password, obtains the data to be processed from the data source corresponding to the account and password, determines the type of the task corresponding to the data to be processed according to the type of the obtained data to be processed as the specified type, where the type of the task includes at least one of the following: Simple type, Stream type, Script type; obtain the task corresponding to the specified type according to the data to be processed; process the task to obtain the processing result.

[0087] In an optional embodiment of this specification, the second data - processing device may further include a front - end page generation module 502, configured to: generate a front - end page and obtain the specified file uploaded through the front - end page.

[0088] In an optional embodiment of this specification, the second data - processing device may further include a second registration module, configured to register the second interface information of the interactive server to the service discovery platform, where the second interface information is used to implement the communication connection with the data - processing end.

[0089] In an optional embodiment of this specification, the front - end page generation module 502 is specifically configured to: receive the data - source information template of the data - processing end, where the data - source information template includes several fields for adding an account and a password; generate a front - end page, where the front - end page displays the data - source information template.

[0090] As Figure 6 shown, an embodiment of the present application provides a data processing device, including a processor 111, a communication interface 112, a memory 113, and a communication bus 114. Among them, the processor 111, the communication interface 112, and the memory 113 complete communication with each other through the communication bus 114.

[0091] The memory 113 is used to store computer programs.

[0092] In an embodiment of the present application, when the processor 111 is used to execute the program stored on the memory 113, it implements the data processing control method provided in any one of the foregoing method embodiments.

[0093] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of data processing provided in any one of the foregoing method embodiments.

[0094] It should be noted that in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element.

[0095] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A data processing method, characterized in that, The method is applied to a data processing end, and the method includes: When receiving a specified file from an interaction server, parsing an account and a password for accessing a data source from the specified file, where the data source stores data to be processed; Using the account and the password to establish a connection with the data source; where an executable file of an application client is parsed from the specified file, the executable file is run, and a connection with the application client is established using the account and the password parsed from the specified file, and the application client is the data source; According to the type of the data to be processed in the data source, determining the type of the task corresponding to the data to be processed as a specified type, where the type of the task includes at least one of the following: Simple type, Stream type, Script type; Obtaining a task corresponding to the specified type according to the data to be processed; Processing the task to obtain a processing result.

2. The method according to claim 1, wherein The specified file includes a jar package, and / or Before receiving the specified file from the interaction server and parsing an account and a password for accessing the data source from the specified file, the method further includes: registering first interface information of the data processing end to a service discovery platform, where the first interface information is used to implement a communication connection with the interaction server.

3. The method according to claim 1, wherein The data processing end is based on a task processing system, and the task processing system includes a registration center and a plurality of nodes. Wherein, processing the task to obtain a processing result includes: A master node among the plurality of nodes performs sharding processing on the task to obtain a plurality of shards; The master node persists the shard marks of the plurality of shards to the registration center; Each node among the plurality of nodes grabs the shard corresponding to the shard mark in the registration center as a target shard for each shard mark in the registration center; If the status of the target shard is in running, processing the target shard using a processing method corresponding to the type of the task to which the target shard belongs to obtain a sub-processing result corresponding to the target shard; Summarizing the sub-processing results corresponding to each shard to obtain a processing result.

4. The method according to claim 1, wherein After detecting the specified file and parsing an account and a password for obtaining the data to be processed from the specified file, the method further includes: if receiving new account and password sent by the interaction server, updating the account and password parsed from the specified file according to the new account and password.

5. The method according to claim 1, characterized in that Before obtaining the specified file, the method further includes: Sending a data source information template to the interaction server, so that the interaction server generates a front-end page displaying the data source information template, where the data source information template includes a plurality of fields for adding an account and a password.

6. A data processing method, characterized in that, The method is applied to an interaction server, and the method includes: When the specified file is obtained, send the specified file to the data processing end, so that the data processing end parses the account number and password for obtaining the data to be processed from the specified file, and then according to the account number and password, obtain the data to be processed from the data source corresponding to the account number and password, and determine the type of the task corresponding to the data to be processed according to the type of the obtained data to be processed, as the specified type, the type of the task includes at least one of the following: Simple type, Stream type, Script type; obtain the task corresponding to the specified type according to the data to be processed; process the task to obtain a processing result; Wherein, the data processing end parses the executable file of the application client from the specified file, runs the executable file, and establishes a connection with the application client by using the account number and password parsed from the specified file, and the application client is the data source.

7. The method according to claim 6, wherein Before sending the specified file to the data processing end when the specified file is obtained, the method further includes: Generate a front-end page and obtain the specified file uploaded through the front-end page; and / or, Register the second interface information of the interaction server to the service discovery platform, where the second interface information is used to implement a communication connection with the data processing end.

8. The method according to claim 7, wherein Generating a front-end page includes: Receive the data source information template of the data processing end, where several fields for adding an account number and a password are included in the data source information template; Generate a front-end page, where the data source information template is displayed in the front-end page.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete communication with each other through the communication bus; The memory is used to store a computer program; The processor is used to implement the steps of the data processing method according to any one of claims 1-8 when executing the program stored on the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the data processing method according to any one of claims 1-8.

Citation Information

Patent Citations

  • Data processing method and device, equipment and storage medium

    CN110472102A

  • Universal distributed heterogeneous data integrated physical convergence, organization, release and service method and system

    CN111259006A

  • Batch processing operation processing method and device

    CN113254437A