Data processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: creating a write-in node under the condition of receiving at least one piece of original data sent by a data sender, and writing the at least one piece of original data into a data storage unit by adopting the write-in node; after writing of the original data is completed, a target cross-tenant node is allocated to a target object, and the target object is a data receiver of the original data; the target cross-tenant node is adopted to send notification information to the target object, so that the target object can obtain original data from the data storage unit according to the notification information; wherein the notification information comprises completion information of writing the original data into the data storage unit by the writing node and / or identification information of the original data. According to the application, the data pushing efficiency and accuracy can be remarkably improved, and human resources are effectively saved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a data processing method, device, electronic device and computer-readable storage medium in the field of data processing technology. Background Art

[0002] At present, data push technology is widely used in various business scenarios to achieve real-time transmission of data in business systems to target systems or end users. However, the traditional data push process often relies on developers to manually organize and send data. This manual process not only consumes a lot of development resources, but is also prone to errors caused by human factors, especially when processing massive amounts of data, resulting in low efficiency and poor accuracy of data push. Summary of the invention

[0003] The present application provides a data processing method, device, electronic device and computer-readable storage medium, which can improve the efficiency and accuracy of data push and save human resources.

[0004] In a first aspect, a data processing method is provided, the method comprising: receiving at least one original data sent by a data sender; creating a write node, and using the write node to write at least one original data into a data storage unit; in response to a data write completion operation of the write node, allocating a target cross-tenant node to a target object; wherein the target object is a data recipient that receives at least one original data; using the target cross-tenant node to send notification information to the target object, so that the target object obtains the original data from the data storage unit according to the notification information; wherein the notification information includes completion information of the write node writing the original data into the data storage unit and / or identification information of the original data.

[0005] Through the above technical solution, the present application creates a write node when receiving at least one original data sent by the data sender, so as to use the write node to write at least one original data into the data storage unit; after the original data is written, the target object, that is, the data recipient of the original data, is assigned a target cross-tenant node; the target cross-tenant node is used to send notification information to the target object, so that the target object can obtain the original data from the data storage unit according to the notification information; wherein the notification information includes the completion information of the write node writing the original data into the data storage unit and / or the identification information of the original data. Compared with the cumbersome process of traditional manual processing and sending data, the present application can significantly improve the efficiency and accuracy of data push and effectively save human resources.

[0006] In combination with the first aspect, in some possible implementations, creating a write node, and using the write node to write at least one original data into a data storage unit includes: for each original data, creating a write node for the original data; and, for the write node created for the original data, using the write node to write the original data into the data storage unit. Based on the above technical solution, by creating a write node for each original data and using these nodes to write the data into the data storage unit, data processing can be parallelized, that is, multiple nodes can process different original data at the same time, thereby speeding up data processing, helping to reduce data delay and improve data processing efficiency.

[0007] In combination with the first aspect and the above implementation, in some possible implementations, creating a write node for the original data includes: calling a table creation statement in a software development kit to create an empty node for the original data; adding an insert statement in the software development kit to the empty node to obtain a write node corresponding to the original data. Based on the above technical solution, the data write node can be quickly created and filled with table creation statements and insert statements in the software development kit, which can significantly improve the speed of data processing.

[0008] In combination with the first aspect and the above implementation, in some possible implementations, using a write node to write raw data into a data storage unit includes: using a table creation statement in the write node to create a target data structure in the data storage unit, and the target data structure can be recognized by the target object; using an insert statement in the write node to add the raw data to the target data structure to write the raw data into the data storage unit. Based on the above technical solution, by creating a target data structure, the raw data can be converted into a format that can be received by the target object, and then the processed data is written into the data storage unit, which can ensure the consistency and standardization of the data and facilitate the target object to perform data query and analysis.

[0009] In combination with the first aspect and the above implementation, in some possible implementations, each raw data carries the object information of the target object receiving the raw data; assigning a target cross-tenant node to the target object includes: creating an initial cross-tenant node corresponding to the write node; configuring the object information on the initial cross-tenant node to obtain the target cross-tenant node; assigning the target cross-tenant node to the target object. Based on the above technical solution, by assigning an independent cross-tenant node to each target object, it can ensure that the data of different target objects are logically isolated, improve data security, and realize multiple nodes to process data in parallel, thereby speeding up data processing.

[0010] In combination with the first aspect and the above implementation, in some possible implementations, creating an initial cross-tenant node corresponding to the write node includes: calling a function in a software development kit for creating a cross-tenant node to create an initial cross-tenant node corresponding to the write node. Based on the above technical solution, by calling a function in a software development kit for creating a cross-tenant node to create an initial cross-tenant node corresponding to the write node, the creation process of the cross-tenant node can be simplified, the complexity of manual configuration can be reduced, the cross-tenant node can be created quickly, and the creation efficiency can be improved.

[0011] In combination with the first aspect and the above implementation, in some possible implementations, a data push application is logged in using robotic process automation to enable the data push application; the data push application is used to execute the step of receiving at least one original data sent by the data sender and sending notification information to the target object using the target cross-tenant node assigned to the target object, so that the target object can obtain the original data from the data storage unit according to the notification information. Based on the above technical solution, by using robotic process automation to log in to the data push application and enabling the data push application to perform the tasks of receiving data and sending notifications, the automation level of data processing can be significantly improved, and human resources can be effectively saved; and errors caused by human factors can be reduced, and the efficiency and accuracy of data processing can be improved.

[0012] In a second aspect, a data processing device is provided, the device comprising:

[0013] A receiving module, used for receiving at least one original data sent by a data sender;

[0014] A processing module is used to create a write node, use the write node to write at least one original data into a data storage unit; and, in response to the data writing completion operation of the write node, allocate a target cross-tenant node to a target object; wherein the target object is a data recipient that receives at least one original data; use the target cross-tenant node allocated to the target object to send notification information to the target object, so that the target object can obtain the original data from the data storage unit according to the notification information; wherein the notification information includes completion information of the write node writing the original data into the data storage unit and / or identification information of the original data.

[0015] In conjunction with the second aspect, in some possible implementations, the processing module includes:

[0016] A first creation unit is used to create a write node for each original data; and

[0017] The writing unit is used to write the original data into the data storage unit by using the writing node created for the original data.

[0018] In combination with the second aspect and the foregoing implementation manner, in some possible implementation manners, the first creation unit includes:

[0019] The first creation subunit is used to call the table creation statement in the software development kit to create an empty node for the original data;

[0020] The first insert subunit is used to add the insert statement in the software development kit to the empty node to obtain the write node corresponding to the original data.

[0021] In combination with the second aspect and the above implementation manner, in some possible implementation manners, the above writing unit includes:

[0022] A second creation subunit is used to create a target data structure in the data storage unit by using the table creation statement in the write node, and the target data structure can be identified by the target object;

[0023] The second insert subunit is used to add the original data to the target data structure by using the insert statement in the write node, so as to write the original data into the data storage unit.

[0024] In combination with the second aspect and the above implementation, in some possible implementations, the processing module further includes:

[0025] A second creation unit, used to create an initial cross-tenant node corresponding to the write node;

[0026] A configuration unit is used to configure the object information on the initial cross-tenant node to obtain a target cross-tenant node; and assign the target cross-tenant node to the target object; wherein each original data carries the object information of the target object receiving the original data.

[0027] In combination with the second aspect and the foregoing implementation manner, in some possible implementation manners, the second creation unit includes:

[0028] The third creation subunit is used to call the function for creating a cross-tenant node in the software development kit to create an initial cross-tenant node corresponding to the write node.

[0029] In combination with the second aspect and the above implementation manner, in some possible implementation manners, the data processing device further includes:

[0030] A control module is used to use robotic process automation to log in to a data push application to enable the data push application; and, use the data push application to execute the steps of receiving at least one original data sent by a data sender and sending notification information to the target object using a target cross-tenant node allocated to the target object, so that the target object can obtain the original data from a data storage unit according to the notification information.

[0031] According to a third aspect, an electronic device is provided, the electronic device comprising:

[0032] A memory for storing executable program codes;

[0033] The processor is used to call and run the executable program code from the memory, so that the electronic device executes the data processing method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0034] In a fourth aspect, a computer program product is provided, which includes: a computer program code, which, when executed on a computer, enables the computer to execute the data processing method in the first aspect or any possible implementation of the first aspect.

[0035] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the data processing method in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic flow chart of a data processing method provided in an embodiment of the present application is shown;

[0037] Figure 2 A schematic flow chart of another data processing method provided in an embodiment of the present application is shown;

[0038] Figure 3 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application is shown;

[0039] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0040] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0041] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0042] At present, data push technology is widely used in various business scenarios to achieve real-time transmission of data in business systems to target systems or end users. However, the traditional data push process often relies on developers to manually organize and send data. This manual process not only consumes a lot of development resources, but is also prone to errors caused by human factors, especially when processing massive amounts of data, resulting in low efficiency and poor accuracy of data push.

[0043] Based on the problems existing in the above-mentioned related technologies, the embodiments of the present application provide a data processing method, device, electronic device and computer-readable storage medium. The present application realizes the complete automation of data sorting and sending by automatically generating write nodes and target cross-tenant nodes during the data push process, thereby improving the efficiency and accuracy of data push and effectively saving human resources.

[0044] The following is an embodiment of a data processing method provided in an embodiment of the present application.

[0045] Figure 1 A schematic flow chart of a data processing method provided in an embodiment of the present application is shown, such as Figure 1 As shown, the above data processing method includes the following schemes:

[0046] S110: Receive at least one original data sent by a data sender.

[0047] In the embodiments of the present application, the data sender refers to the party that sends the data, that is, the entity responsible for generating or collecting data and sending the data to the recipient. The data sender can be various types of devices or systems, such as computer systems, smart phones, and database management systems. The role of the data sender is to ensure that the data is correctly packaged and sent to the designated data storage unit, which includes, for example, a data lake and downstream systems for further processing and analysis. Raw data refers to data obtained directly from the source by the data sender without any processing or conversion.

[0048] For example, suppose there is a sales data analysis system that needs to obtain sales data provided by the sales system's database and store it in the data lake for further analysis. In this embodiment, the data sender is the sales system's database, which regularly sends sales data to the data lake for subsequent data analysis and report generation. The sales data provided by the sales system's database is the original data.

[0049] S120: Create a write node, and use the write node to write at least one original data into a data storage unit.

[0050] Creating a write node and using the write node to write at least one raw data into a data storage unit can achieve full automation of data sorting and sending, and improve the efficiency and accuracy of data push.

[0051] In one possible implementation, a write node is created and at least one original data is written into a data storage unit using the write node, including: for each original data, creating a write node for the original data; and for the write node created for the original data, using the write node to write the original data into the data storage unit.

[0052] It should be understood that the write node is used to receive the original data and perform preliminary preprocessing on it, such as data cleaning, format conversion, etc., and write the processed original data into the data storage unit. For each original data, a write node is assigned to each original data. On the one hand, the processing logic can be customized according to the characteristics of the data or the needs of the user, making the data processing more flexible, helping to adapt to the changes in different data types and structures, and enabling the system to better cope with diverse data sources; on the other hand, creating a write node for each original data and using these nodes to write the data into the data storage unit can achieve parallelization of data processing, that is, multiple nodes can process different original data at the same time, thereby speeding up the data processing speed, helping to reduce data delay and improve the efficiency of data processing. The following describes the process of creating a write node by taking the creation of a write node corresponding to the first original data in at least one original data as an example, and the first original data is any original data in at least one original data.

[0053] In one possible implementation, creating a write node for the first original data includes: calling a table creation statement in a software development kit to create an empty node for the first original data; adding an insert statement in the software development kit to the empty node to obtain a write node corresponding to the first original data.

[0054] It should be understood that the empty node created by calling the table building statement in the software development kit does not contain the first original data. The insert statement in the software development kit is added to the empty node to obtain the first write node, and the first write node is the write node corresponding to the first original data. Wherein, the effect of the insert statement is to write the first original data to the data storage unit. Exemplarily, the development tool for creating the write node can be a software development kit (Software Development Kit, SDK), or other types of software development tools. Which type of software development kit is selected depends on the needs of different developers, and this application is not limited to this.

[0055] Based on the above technical solution, the speed of data processing can be significantly improved by quickly creating and filling write nodes through table creation statements and insert statements in the software development kit.

[0056] Exemplarily, after the write node corresponding to the first original data is created, the first write node is used to write the first original data into the data storage unit.

[0057] In one possible implementation, the first original data is written into the data storage unit using the first write node, including: using the table creation statement in the first write node to create a first target data structure in the data storage unit, and the first target data structure can be recognized by the first target object; using the insert statement in the first write node to add the first original data to the first target data structure to write the first original data into the data storage unit.

[0058] Among them, the first target object is a data recipient who receives the first original data. The first target data structure refers to a data organization form created in the data storage unit, which can be recognized and used by the first target object. In the embodiment of the present application, the target data structure has identification information for uniquely identifying each original data. The target data structure can be a database table or other types of data containers, such as documents, key-value pairs, etc. The type of target data structure selected depends on the needs of the target object, and the present application does not limit this.

[0059] For example, in the above-mentioned sales data analysis embodiment, the first original data is sales data, the identification information of the first original data is b001, and the first original data includes the following fields: id, product ID (product_id), sales date (sales_date), sales quantity (quantity) and unit price (price), among which id is the primary key field, which is used to uniquely identify each record, and the first target data structure is shown in Table 1.

[0060] Table 1

[0061]

[0062] It should be understood that before the insert statement in the first write node is used to add the first original data to the first target data structure, the preprocessing statement in the first write node needs to be used to clean the first original data to ensure the accuracy and integrity of the data. For example, data cleaning includes deleting data that does not meet the requirements in the first original data, and / or sorting the first original data to ensure that the first original data is arranged in a certain order.

[0063] After the first original data is cleaned, the cleaned first original data is added to the first target data structure to write the first original data into the data storage unit. Exemplarily, the first target data structure to which the first original data is added is shown in Table 2.

[0064] Table 2

[0065]

[0066] Exemplarily, the first target data structure to which the first original data is added includes two sales data records, namely, the id, product ID, sales date, sales quantity and unit price of product P001 and product P002.

[0067] Based on the above technical solution, by using table creation statements to create a target data structure that can be recognized by the target object, the original data can be converted into a format that can be received by the target object. By using insert statements to write the processed data into the data storage unit, the consistency and standardization of the data can be ensured, making it easier for the target object to query and analyze the data.

[0068] S130: In response to the data writing operation of the write node being completed, a target cross-tenant node is allocated for the target object.

[0069] In an embodiment of the present application, in response to the data write completion operation of the write node indicating that the write node has written the data into the data storage unit, a notification message can be started to be sent to the target object. The target cross-tenant node is used to send a notification message to the target object to remind the target object that the data has been written to the data storage unit and can be used. Therefore, after the data write operation of the write node is completed, an embodiment of the present application allocates a target cross-tenant node to the target object so as to use the target cross-tenant node to send a notification message to the target object. The following takes the allocation of a first target cross-tenant node to a first target object that receives the first original data as an example to describe the process of allocating a target cross-tenant node to a target object.

[0070] Exemplarily, the first original data carries object information of a first target object that receives the first original data, and a target cross-tenant node is assigned to the first target object, including: creating an initial cross-tenant node corresponding to the first write node, referred to as a first initial cross-tenant node; configuring the object information of the first target object on the first initial cross-tenant node to obtain a first target cross-tenant node; and assigning the first target cross-tenant node to the first target object.

[0071] It should be understood that the created initial cross-tenant node is configured with parameters such as the call time, but does not contain the object information of the target object. Configuring the object information of the target object for the initial cross-tenant node can send the notification information to the target object using the target cross-tenant node according to the object information of the target object after the data write operation of the write node is completed. Among them, the object information includes the account number, password, and address of the target object. Exemplarily, the object information of the first target object is configured in the first initial cross-tenant node to obtain the first target cross-tenant node; the first target cross-tenant node is assigned to the first target object, so that a corresponding relationship is established between the first target cross-tenant node and the first target object, ensuring that the first target cross-tenant node only serves the first target object, thereby achieving a high degree of personalization and security of data processing and resource management.

[0072] In a possible implementation, creating a first initial cross-tenant node corresponding to the first write node includes: calling a function for creating a cross-tenant node in a software development kit to create the first initial cross-tenant node corresponding to the first write node.

[0073] For example, the function for creating a cross-tenant node in the software development kit encapsulates all the logic and steps required to create a cross-tenant node, and can quickly create a cross-tenant node according to preset parameters, thereby greatly reducing the complexity of manual configuration and simplifying the entire creation process to improve creation efficiency. In addition, the software development kit usually provides highly configurable functions that can flexibly customize the configuration of the cross-tenant node according to different needs to meet specific application scenarios.

[0074] Exemplarily, in some other embodiments of the present application, the same number of initial cross-tenant nodes as the number of target objects can be established according to the number of target objects, and an initial cross-tenant node list can be output; the initial cross-tenant nodes in the initial cross-tenant node list can be opened in sequence, and according to the unique identification information of each original data, the information of the target object associated with the identification information can be configured to the initial cross-tenant node corresponding to the identification information, so as to obtain the target cross-tenant node corresponding to each target object. Exemplarily, the initial cross-tenant node list is shown in Table 3, which contains the identification information of at least one original data and the initial cross-tenant node.

[0075] Table 3

[0076] Identification Information Initial cross-tenant node b001 Node A b002 Node B ... ...

[0077] For example, the identification information of the original data 1 is b001, and the target object receiving the original data 1 is object 1, that is, the object information of object 1 is configured on node A corresponding to the identification information b001, and the target cross-tenant node corresponding to object 1 is obtained, that is, node A configured with the object information; so that node A configured with the object information can be used to send notification information to object 1.

[0078] Based on the above technical solution, by allocating an independent cross-tenant node to each target object, it can ensure that the data of different target objects are logically isolated, thereby improving data security. At the same time, multiple nodes can process data in parallel, thereby speeding up data processing.

[0079] S140: Use the target cross-tenant node to send notification information to the target object, so that the target object obtains original data from the data storage unit according to the notification information.

[0080] The notification information includes completion information of the writing node writing the original data into the data storage unit and / or identification information of the original data. It should be understood that the completion information of the writing node writing the original data into the data storage unit is used to indicate that the data required by the target object has been written into the data storage unit, and the target object can obtain the original data from the data storage unit according to the identification information of the original data and apply it.

[0081] It should be understood that the execution order of the write node and the cross-tenant node is that the write node completes data writing first. After the data writing is completed, the write node sends a reminder to the cross-tenant node to remind the cross-tenant node that the data writing is completed, and the cross-tenant node performs the operation of sending notification information at a preset time point; that is, the execution of the cross-tenant node is subject to two conditions: one is the receipt of the data writing completion reminder from the write node, and the other is the preset sending time point, that is, before receiving the reminder, even if the preset notification time point is reached, the cross-tenant node will not send any message; or, the data writing completion reminder from the write node has been received, but the preset time point has not been reached, and the cross-tenant node will not send notification information; that is, the cross-tenant node will send notification information to the target object only when the two conditions of receiving the data writing completion reminder from the write node and reaching the preset time point are met at the same time.

[0082] For example, suppose there is a data push task, the preset time point is 12 o'clock, the write node is responsible for receiving data and writing it into the data storage unit, and the cross-tenant node is used to send a notification message to the target object to inform it that the data has been written; if the write node completes the data write operation at 11 o'clock and sends a data write completion reminder to the cross-tenant node; after receiving the data write completion reminder, the cross-tenant node determines whether the preset time point has been reached. Since the preset sending time point of the cross-tenant node is 12 o'clock, the preset time point has not been reached; therefore, the cross-tenant node does not perform the operation of sending the notification message, and continues to wait until the preset time point is reached, that is, at 12 o'clock, and performs the operation of sending the notification message to the target object; or, if the preset time point is reached, that is, at 12 o'clock in the evening, but the cross-tenant node has not received the data write completion reminder sent by the write node, the cross-tenant node does not perform the operation of sending the notification message, and continues to wait until the data write completion reminder is received, and in response to the data write completion signal, sends the notification message to the target object.

[0083] It should be understood that since the cross-tenant node sends notification information after the write node is completed and when the preset time point is reached, it can ensure that the notification is sent only after the data is written, avoiding the data inconsistency problem that may be caused by early sending; and sending notification information in accordance with the preset time point can ensure the timeliness and regularity of the message, thereby improving the stability and efficiency of the system.

[0084] Through the above technical solution, this application automatically generates write nodes and target cross-tenant nodes during the data push process, realizing complete automation of data collation and sending; compared with the cumbersome process of traditional manual processing and sending of data, this application can significantly improve the efficiency and accuracy of data push and effectively save human resources.

[0085] The above is a solution flow of a data processing method provided in an embodiment of the present application. The following describes the application process of a data processing method provided in an embodiment of the present application in combination with a specific application scenario.

[0086] In some possible implementations, robotic process automation is used to log in to a data push application to enable the data push application; the data push application is used to execute the steps of receiving at least one original data sent by a data sender and sending a notification message to the target object using a target cross-tenant node assigned to the target object, so that the target object can obtain the original data from the data storage unit according to the notification message.

[0087] For example, Figure 2A schematic flow chart of another data processing method provided by an embodiment of the present application is shown. It should be understood that a data processing method provided by an embodiment of the present application is applied to a data push application, and the present application uses a robotic process automation to log in to the data push application to enable the data push application; and the process of using the data push application to execute the data processing method includes the following steps:

[0088] S201: Robotic process automation login data push application.

[0089] In traditional data push operations, the data integration, sorting and sending processes often rely entirely on manual operations. This data push method is not only cumbersome and inefficient, but also very easy to cause errors due to human factors when processing massive amounts of data. In order to significantly improve the efficiency and accuracy of data push, this application uses robotic process automation technology, so that the data integration, sorting and sending links that originally required manual intervention are now completely completed by automated robots. Through the Robotics Process Automation (RPA) technology to learn human operations, log in and use the data push application, the data push operation is fully automated, which greatly improves work efficiency, effectively reduces the risk of human errors, and ensures the accuracy and timeliness of data push.

[0090] S202: Obtain a data push list.

[0091] Exemplarily, after logging into the data push application, a data push list that needs to be processed is displayed in the interactive page of the data push application. The data push list is a list of all data push tasks to be processed, and the list contains at least one original data sent by at least one data sender. For example, the data push list may be as shown in Table 4; wherein the data push list shown in Table 4 includes 4 push tasks, each of which contains the information of the data sender and the original data to be pushed. Among them, the task number is equivalent to the identification information of the original data. For example, the data senders in the data push list shown in Table 4 include the sales department, the purchasing department, customer A and customer B; the original data provided by the sales department is sales report 1, the original data provided by the purchasing department is purchase list 1, the original data provided by customer A is customer feedback 1, and the original data provided by customer B is customer feedback 2.

[0092] Table 4

[0093] Task Number Data sender Raw data 001 Sales Department Sales Report 1 002 Purchasing Department Shopping List 1 003 Customer A Customer feedback 1 004 Customer B Customer Feedback 2

[0094] S203: Call the SDK to read the data push list.

[0095] The SDK can provide interfaces or functions for accessing and processing data sources to extract necessary metadata and other relevant information from the data push list, such as column names, data types, data volumes, data formats, etc. Among them, the purpose of reading the data push list is to better understand and process the data. It is convenient for subsequent data cleaning, conversion, and loading. Exemplarily, the process of calling the SDK to read the data push list shown in Table 4 may include the following steps: 1. Connect to the data source: Use the functions or interfaces provided by the SDK to connect to Table 4; 2. Read metadata: Read the metadata of Table 4, such as column names, data types, data volumes, etc., to understand the structure and content of the data; 3. Parse data: Parse the contents of Table 4 and convert it into a processable form, including converting the data into arrays, dictionaries, or other data structures; 4. Save detailed information: Save the obtained detailed information of Table 4 for subsequent use, including saving the metadata to a database or file.

[0096] S204: Calling a table creation statement and an insert statement to create a write node.

[0097] Exemplarily, after reading the contents of the data push list shown in Table 4, the table creation statement in the software development kit is called to create an empty node for each of the four original data in the data push list shown in Table 4; the empty nodes corresponding to the four original data do not contain the corresponding original data, and the insert statements in the software development kit are added to the four empty nodes respectively to obtain the write nodes corresponding to the four original data.

[0098] S205: Call the SDK to fill in the scheduling information and submit the write node.

[0099] After creating the write nodes corresponding to the four original data, call the SDK to set the scheduling information and submit the write node to the system of the data push application, which means that the scheduling information of the write node is defined through the interface or function provided by the SDK. The scheduling information includes parameters such as the time, frequency, and priority of the task execution. These parameters determine when the task of the write node starts to execute and the periodicity of the execution; after the scheduling information is set, the write node is added to the system, that is, the write node is used as part of the data processing flow.

[0100] Exemplarily, after the scheduling information of the write nodes corresponding to the four original data is set, the write nodes corresponding to the four original data are used to write the four original data into the data storage unit according to the scheduling information of the write nodes corresponding to the four original data. Specifically, the table creation statement in the write nodes corresponding to the four original data is used to create the target data structures corresponding to the four original data in the data storage unit, wherein the four target data structures are data structures that can be received by the target objects corresponding to the four original data; the insert statement in the write nodes corresponding to the four original data is used to add the cleaned four original data to the corresponding target data structures, so as to write the four original data into the data storage unit.

[0101] S206: Call the SDK to create an initial cross-tenant node and fill in scheduling information.

[0102] After writing the four original data into the data storage unit respectively, call the function for creating cross-tenant nodes in the SDK to create the initial cross-tenant nodes corresponding to the four original data respectively; after the initial cross-tenant nodes corresponding to the four original data are created, define the scheduling information of the cross-tenant nodes through the interface or function provided by the SDK. The scheduling information includes parameters such as the time, frequency, and priority of task execution. These parameters determine when the cross-tenant node tasks start to execute and the periodicity of the execution.

[0103] S207: Output an initial cross-tenant node list.

[0104] Exemplarily, the initial cross-tenant node list is shown in Table 5, and the initial cross-tenant node list includes the identification information of the above four original data and the initial cross-tenant nodes corresponding to each of the four original data.

[0105] Table 5

[0106] Identification Information Initial cross-tenant node 001 Node 1 002 Node 2 003 Node 3 004 Node 4

[0107] S208: Read the initial cross-tenant node.

[0108] Read the initial cross-tenant nodes in the initial cross-tenant node list in sequence.

[0109] S209: Determine whether the initial cross-tenant node is ready.

[0110] Determine whether the initial cross-tenant node meets the ready condition, wherein the ready condition includes: the initial cross-tenant node can be searched, and the initial cross-tenant node can receive and send data normally. If both of the above conditions are met, it is determined that the initial cross-tenant node is ready, and S210 is executed; if any of the above two conditions is not met, it is determined that the initial cross-tenant node is not ready, and S211 is executed. For example, determine whether node 1 shown in Table 5 is ready, including: searching for node 1, if node 1 can be searched, then check whether node 1 can receive and send data normally, if node 1 can receive and send data normally, then determine that node 1 is ready, and execute S210; if node 1 cannot be searched or node 1 fails and cannot receive and send data normally, then determine that node 1 is not ready, and execute S211.

[0111] S210: Open the initial cross-tenant node and fill in the account information.

[0112] It should be understood that the parameters such as the call time are configured in the created initial cross-tenant node, but the object information of the target object is not included. When it is determined that the initial cross-tenant node is ready, the initial cross-tenant node is opened, and the target object of the original data corresponding to the identification information is determined based on the identification information of the initial cross-tenant node, and the information of the target object is configured in the initial cross-tenant node to obtain the target cross-tenant node corresponding to the target object.

[0113] S211: Record failure log.

[0114] If it is determined that the node is not ready, the factors that cause the node to be unready are recorded to identify and solve potential problems. For example, if the factor that causes node 1 to be unready is that node 1 cannot be searched, the possible reasons include network problems, configuration errors, or node 1 does not exist. You can check the network connection, configuration file, or whether node 1 exists to solve the problem that node 1 is not ready; or, if the factor that causes node 1 to be unready is that node 1 cannot receive and send data normally, there may be a problem with node 1 itself, and you can solve the problem that node 1 is not ready by checking the health status, resource limitations, or data format of node 1; and after the recording is completed, return to execute S208.

[0115] S212: Submit and go online.

[0116] Submitting online means submitting the cross-tenant node configured with object information to the system, that is, writing the node as part of the data processing flow, so that the system can use the target cross-tenant node to send notification information to the target object.

[0117] Exemplarily, after the cross-tenant node configured with object information is submitted to the system, the cross-tenant node is used to send notification information to the target object corresponding to the original data, so that the target object can obtain the original data from the data storage unit according to the notification information. The notification information includes the completion information of the writing node writing the original data into the data storage unit and / or the identification information of the original data; the completion information of the writing node writing the original data into the data storage unit is used to indicate that the data required by the target object has been written into the data storage unit, and the target object can obtain the original data from the data storage unit according to the identification information of the original data and apply it.

[0118] It should be understood that the execution of the cross-tenant node is subject to two conditions: one is the receipt of the data writing completion reminder from the write node, and the other is the configured execution time point. For example, the execution time point configured for the target cross-tenant node corresponding to the original data sales report 1 in the data push list shown in Table 4 is 9 o'clock. After the write node corresponding to sales report 1 completes the writing operation of sales report 1, the write node sends a data writing completion reminder to the cross-tenant node to indicate that the writing of sales report 1 has been completed. After receiving the data writing completion reminder, the cross-tenant node corresponding to sales report 1 executes the operation of sending notification information to the target object corresponding to sales report 1 at 9 o'clock.

[0119] S213: Determine whether the initial cross-tenant node list has been read.

[0120] After configuring the object information for each cross-tenant node, determine whether all initial cross-tenant nodes in the initial cross-tenant node list have been configured. If so, execute S214; if not, return to execute S208.

[0121] S214: Close the data push application.

[0122] After all initial cross-tenant nodes in the initial cross-tenant node list are configured, that is, after all push tasks in the data push list shown in Table 4 are processed, the data push application is closed.

[0123] Based on the above technical solution, by using robotic process automation to log in to the data push application and enabling the data push application to perform the task of generating write nodes and target cross-tenant nodes, complete automation of data collation and sending is achieved, effectively saving human resources; and it can reduce errors caused by human factors and improve the efficiency and accuracy of data processing.

[0124] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0125] Figure 3FIG. 1 is a schematic diagram showing the structure of a data processing device provided in an embodiment of the present application. Figure 3 As shown, the data processing device 300 includes:

[0126] The receiving module 310 is used to receive at least one original data sent by a data sender;

[0127] Processing module 320 is used to create a write node, use the write node to write at least one original data into a data storage unit; and, in response to the data writing completion operation of the write node, allocate a target cross-tenant node to a target object; wherein the target object is a data recipient that receives at least one original data; use the target cross-tenant node allocated to the target object to send notification information to the target object, so that the target object can obtain the original data from the data storage unit according to the notification information; wherein the notification information includes completion information of the write node writing the original data into the data storage unit and / or identification information of the original data.

[0128] In a possible implementation, the processing module 320 includes:

[0129] A first creation unit is used to create a write node for each original data; and

[0130] The writing unit is used to write the original data into the data storage unit by using the writing node created for the original data.

[0131] In a possible implementation, the first creation unit includes:

[0132] The first creation subunit is used to call the table creation statement in the software development kit to create an empty node for the original data;

[0133] The first insert subunit is used to add the insert statement in the software development kit to the empty node to obtain the write node corresponding to the original data.

[0134] In a possible implementation, the writing unit includes:

[0135] A second creation subunit is used to create a target data structure in the data storage unit by using the table creation statement in the write node, and the target data structure can be identified by the target object;

[0136] The second insert subunit is used to add the original data to the target data structure by using the insert statement in the write node, so as to write the original data into the data storage unit.

[0137] In a possible implementation, the processing module 320 further includes:

[0138] A second creation unit, used to create an initial cross-tenant node corresponding to the write node;

[0139] A configuration unit is used to configure the object information on the initial cross-tenant node to obtain a target cross-tenant node; and assign the target cross-tenant node to the target object; wherein each original data carries the object information of the target object receiving the original data.

[0140] In a possible implementation, the second creation unit includes:

[0141] The third creation subunit is used to call the function for creating a cross-tenant node in the software development kit to create an initial cross-tenant node corresponding to the write node.

[0142] In a possible implementation, the data processing device 300 further includes:

[0143] A control module is used to use robotic process automation to log in to a data push application to enable the data push application; and, use the data push application to execute the steps of receiving at least one original data sent by a data sender and sending notification information to the target object using a target cross-tenant node allocated to the target object, so that the target object can obtain the original data from a data storage unit according to the notification information.

[0144] Figure 4 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0145] For example, Figure 4 As shown, the electronic device 400 includes: a memory 401 and a processor 402, wherein the memory 401 stores an executable program code 4011, and the processor 402 is used to call and execute the executable program code 4011 to perform a data processing method.

[0146] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be corresponded to, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0147] In the case of dividing each functional module according to each function, the electronic device may include: an acquisition module, a processing module, a control module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0148] The electronic device provided in this embodiment is used to execute the above data processing method, so the same effect as the above implementation method can be achieved.

[0149] In the case of an integrated unit, the electronic device may include a processing module and a storage module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device in executing related program codes and data.

[0150] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0151] This embodiment further provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement a data processing method in the above-mentioned embodiment.

[0152] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a data processing method in the above-mentioned embodiment.

[0153] In addition, the electronic device provided in the embodiments of the present application may specifically be a chip, a component or a module, and the electronic device may include a connected processor and a memory; wherein the memory is used to store instructions, and when the electronic device is running, the processor may call and execute instructions so that the chip executes a data processing method in the above-mentioned embodiments.

[0154] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding data processing method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding data processing method provided above, and will not be repeated here.

[0155] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0156] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0157] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The data processing method comprises: receiving at least one original data sent by a data sender; Creating a write node, and using the write node to write the at least one original data into a data storage unit; In response to the data writing completion operation of the write node, allocating a target cross-tenant node to a target object; wherein the target object is a data recipient that receives the at least one original data; The target cross-tenant node is used to send notification information to the target object, so that the target object obtains the original data from the data storage unit according to the notification information; wherein the notification information includes completion information of the write node writing the original data into the data storage unit and / or identification information of the original data.

2. The data processing method according to claim 1, characterized in that: The creating of the write node and using the write node to write the at least one original data into the data storage unit comprises: For each raw data, create a write node for the raw data; and, For a write node created for the original data, the original data is written into the data storage unit using the write node.

3. The data processing method according to claim 2, characterized in that: The step of creating a write node for the original data includes: Calling a table creation statement in a software development kit to create an empty node for the original data; Add the insert statement in the software development kit to the empty node to obtain the write node corresponding to the original data.

4. The data processing method according to claim 3, characterized in that: The adopting the write node to write the original data into the data storage unit comprises: Using the table creation statement in the write node, a target data structure is created in the data storage unit, and the target data structure can be identified by the target object; The original data is added to the target data structure using an insert statement in the write node to write the original data into the data storage unit.

5. The data processing method according to claim 1, characterized in that: Each original data carries object information of a target object receiving the original data; The allocating a target cross-tenant node to the target object includes: Creating an initial cross-tenant node corresponding to the write node; Configuring the object information on the initial cross-tenant node to obtain the target cross-tenant node; Assign the target cross-tenant node to the target object.

6. The data processing method according to claim 5, characterized in that: The creating an initial cross-tenant node corresponding to the write node includes: A function for creating a cross-tenant node in the software development kit is called to create an initial cross-tenant node corresponding to the write node.

7. The data processing method according to any one of claims 1 to 6, characterized in that: The data processing method further includes: Logging into a data push application using robotic process automation to enable the data push application; The data push application is used to execute the step of receiving at least one original data sent by the data sender and sending notification information to the target object by the target cross-tenant node allocated to the target object, so that the target object can obtain the original data from the data storage unit according to the notification information.

8. A data processing device, characterized in that: The data processing device comprises: A receiving module, used for receiving at least one original data sent by a data sender; A processing module is used to create a write node, use the write node to write the at least one original data into a data storage unit; and, in response to the data writing completion operation of the write node, allocate a target cross-tenant node to a target object; wherein the target object is a data recipient that receives the at least one original data; use the target cross-tenant node allocated to the target object to send notification information to the target object, so that the target object can obtain the original data from the data storage unit according to the notification information; wherein the notification information includes completion information of the write node writing the original data into the data storage unit and / or identification information of the original data.

9. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the electronic device executes the data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the data processing method according to any one of claims 1 to 7 is implemented.

Images