Data processing method and device, storage medium and electronic device

Abstract

The present application discloses a data processing method and device, a storage medium, and an electronic device. The method includes: receiving a data transmission request initiated by a data management system, wherein the data transmission request is used to request the transmission of target data; in response to the data transmission request, detecting whether a list file exists in a network shared directory, wherein the list file is used to indicate the data file to be transmitted, the data file is a file directly uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file; when the list file is detected in the network shared directory, the data file is transmitted to the business platform at a preset time according to the list file. The present application solves the technical problem of low data processing efficiency.

Description

Technical Field

[0001] The present application relates to the field of computers, and more specifically, to a data processing method and device, a storage medium, and an electronic device. Background Art

[0002] At present, in related technologies, system work orders are mainly manually established, and the process approval of each system link is carried out in sequence, so as to achieve the safe transmission of part of the data in the production environment to the application environment for use by the business platform in the application environment. In this process, due to the long time required for the processes of each link and the untimely data transmission, the technical problem of low data processing efficiency is caused.

[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0004] The embodiments of the present application provide a data processing method and device, a storage medium, and an electronic device to at least solve the technical problem of low data processing efficiency.

[0005] According to one aspect of an embodiment of the present application, a data processing method is provided, comprising: receiving a data transmission request initiated by a data management system, wherein the data transmission request is used to request transmission of target data; in response to the data transmission request, detecting whether a list file exists in a network shared directory, wherein the list file is used to indicate data files to be transmitted, the data files including the target data, the data files being files uploaded by the data management system, and / or files obtained by dividing the target data according to the business category of the target data, and the network shared directory being used to store the list file and the data files; in a case where the list file is detected in the network shared directory, transmitting the data files to a business platform at a preset time according to the list file, wherein data isolation exists between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

[0006] According to another aspect of an embodiment of the present application, a data processing device is also provided, including: a receiving module for receiving a data transmission request initiated by a data management system, wherein the data transmission request is used to request the transmission of target data; a detection module for detecting whether there is a list file in a network shared directory in response to the data transmission request, wherein the list file is used to indicate the data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file; a transmission module for transmitting the data file to a business platform at a preset time according to the list file when the list file is detected in the network shared directory, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

[0007] Optionally, the device is also used to: when the list file is detected in the network shared directory, before transmitting the data file to the business platform according to the list file, receive the data file uploaded in the data management system through a secure file transfer protocol, and store the data file in the network shared directory; establish an index of the data file according to the data attributes of the target data to generate the list file; store the list file in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

[0008] Optionally, the device is also used to: when the list file is detected in the network shared directory, before transferring the data file to the business platform according to the list file, determine the target parameters related to the target data in response to the target interaction action occurring in the data management system, wherein the target parameters are used to indicate the data attributes of the target data, and the target interaction operation is used to indicate the acquisition of the target data from the business database; initiate a data acquisition request to the business database according to the target parameters in the data transmission request to acquire the target data in the business database; and generate the data file and the list file according to the target data and the target parameters.

[0009] Optionally, the device is used to generate the data file and the list data according to the target data and the target parameters in the following manner: dividing the target data according to the business category of the target data to obtain the data file; establishing an index of the data file according to the target parameters, generating the list file, and storing the list file in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

[0010] Optionally, the device is also used for at least one of the following: obtaining a list number in the data management system, wherein the list number is used to indicate the upload time and upload type of the data file; receiving the data transmission request initiated by the data management system based on the list number and the transmission node, wherein the transmission node is determined by the business category of the target data; when the target data is in a target state, receiving the data transmission request initiated by the data management system based on the transmission node, wherein the target state indicates that the target data is allowed to be accessed.

[0011] Optionally, the device is also used to: periodically run a file generation script to determine whether the data file exists in the network shared directory, wherein the file generation script is deployed on a target server, and the target server allows access to the network shared directory; when the data file exists in the network shared directory, establish an index of the data file according to the target parameters, generate the list file, and store the list file in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

[0012] Optionally, the device is also used to: periodically run a file detection script to detect whether the list file exists in the network shared directory, wherein the file detection script is deployed on a target server, the target server allows access to the network shared directory, and the target server is used to periodically transmit the data file to the business platform.

[0013] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the above-mentioned data processing method when it is run.

[0014] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above data processing method.

[0015] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the data processing method through the computer program.

[0016] In an embodiment of the present application, a data transmission request initiated by a receiving data management system is adopted, wherein the data transmission request is used to request the transmission of target data; in response to the data transmission request, a list file is detected in the network shared directory, wherein the list file is used to indicate the data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file; in the case where the list file is detected in the network shared directory, the data file is transmitted to the business platform at a preset time according to the list file, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit the initiation of a request to the data management system to obtain the target data, and the purpose of improving data processing is achieved by automating the steps of transmission and verification of the target data. At the same time, the security of the data transmission process is effectively guaranteed, thereby achieving the technical effect of saving data transmission time and simplifying the data processing process, thereby solving the technical problem of low data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a schematic diagram of an application environment of an optional data processing method according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of an optional data processing method according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of an optional data processing method according to an embodiment of the present application;

[0021] Figure 4is a schematic structural diagram of an optional data processing device according to an embodiment of the present application;

[0022] Figure 5 is a schematic structural diagram of an optional data processing product according to an embodiment of the present application;

[0023] Figure 6 It is a schematic structural diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] The present application will be described below with reference to the following embodiments:

[0027] According to one aspect of an embodiment of the present application, a data processing method is provided. Optionally, in this embodiment, the data processing method can be applied to Figure 1 In the hardware environment composed of the server 101 and the terminal device 103 shown in FIG. Figure 1As shown, the server 101 is connected to the terminal device 103 via a network and can be used to provide services for the terminal device or the application installed on the terminal device. The application 107 can be a video application, instant messaging application, browser application, educational application, game application, etc. A database 105 may be set up on the server or independently of the server to provide data storage services for the server 101, for example, a game data storage server. The above-mentioned network may include but is not limited to: a wired network, a wireless network, wherein the wired network includes: a local area network, a metropolitan area network and a wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that realize wireless communication. The terminal device 103 may be a terminal configured with an application, and may include but is not limited to at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop computer, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a mixed reality (MR) terminal and other computer devices. The above-mentioned server may be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0028] Combine Figure 1 As shown, the above data processing method can be executed by an electronic device, which can be a terminal device or a server. The above data processing method can be implemented by the terminal device or the server separately, or by the terminal device and the server together.

[0029] The above is only an example and is not specifically limited in this embodiment.

[0030] Alternatively, as an optional implementation, as Figure 2 As shown, the above data processing method includes:

[0031] S202, receiving a data transmission request initiated by a data management system, wherein the data transmission request is used to request transmission of target data;

[0032] Optionally, in an embodiment of the present application, the above-mentioned target data may include but is not limited to being input by the user in the business application system through a mouse and keyboard, touch screen, voice, etc., and then stored in the business database by the business application system to process the relevant business data; or, the target data may also be some sensitive data generated in the production environment corresponding to the business application system, which can also be stored in the business database.

[0033] It is understandable that the business application system here can refer to different systems in different application environments. For example, taking banking business as an example, it can be a banking business application or banking business web system on a smart terminal, or it can be an internal application development system of the bank, etc. The smart terminals here can include but are not limited to mobile phones, computers, notebooks, smart wearable devices, etc.

[0034] In an exemplary embodiment, a user logs into a banking application using a mobile phone and enters an account number and password on the login interface. Assuming that the user logs in using a mobile phone number and a one-time verification code, the mobile phone number entered by the user is the target data.

[0035] In another exemplary embodiment, the business development department of a bank trains a business prediction model in the bank's internal application development system, and the model parameters corresponding to the model are the above-mentioned target data.

[0036] Furthermore, the above-mentioned data management system is also deployed in an internal production environment and isolated from the external application environment. Users can verify their identity through methods such as data sandbox, log in to the data management system and choose to upload the data file generated after processing the target data, or initiate a request in the data management system to instruct another intelligent management system in the production environment to obtain the target data stored in the business database, and then the intelligent management system processes the target data and generates a data file.

[0037] Furthermore, the above-mentioned data transmission request refers to a request for requesting the transmission of target data, which may include but is not limited to an https request, etc., and its request parameters include those used to indicate the source of the target data, transmission address, data type, data storage location, data transmission protocol, etc.

[0038] S204, in response to the data transmission request, detecting whether a list file exists in the network shared directory, wherein the list file is used to indicate data files to be transmitted, the data files including target data, the data files being files uploaded by the data management system and / or files obtained by classifying the target data according to the service category of the target data, and the network shared directory being used to store the list file and the data files;

[0039] Optionally, in an embodiment of the present application, the above-mentioned data files can be files obtained by dividing and classifying according to the business categories corresponding to each data in the above-mentioned target data, and each data file contains data of the corresponding business field; or, divided according to the data type corresponding to each data in the target data, such as text data files, image data files, audio data files, etc.; or, divided according to the data source corresponding to each data in the target data, such as data files generated by business department A, data files generated by business department B, etc.; or, divided according to the data usage corresponding to each data in the target data, such as data files for analysis, data files for reporting, data files for backup, etc.; or, divided according to the data format corresponding to each data in the target data, such as CSV files, XML files, JSON files, database files, etc.

[0040] In an exemplary embodiment, it is assumed that the above-mentioned data file is a file obtained by dividing and classifying according to the business categories corresponding to each data in the above-mentioned target data. Taking the banking business system as an example, the business categories include but are not limited to deposit business, loan business, investment business, payment business, etc. Specifically, it can be further subdivided into primary business, secondary business belonging to the sub-business of the primary business, etc., and divided and classified according to the sub-business type to which each data belongs. For example, the primary business is deposit business, loan business, investment business, and payment business. The secondary business under the deposit business is current deposits, time deposits, notice deposits, foreign currency deposits, savings deposits, etc.; the secondary business under the loan business is personal loans, corporate loans, credit card loans, mortgage loans, credit loans, etc.; the secondary business under the investment business is stock investment, fund investment, futures investment, foreign exchange investment, etc.; the secondary business under the payment business is bank card payment, e-wallet payment, mobile payment, etc.

[0041] Furthermore, taking the target data as the telephone number and residential address reserved by user A when making a current deposit, the telephone number and residential address reserved by user B when making a current deposit, the telephone number and residential address reserved by user A when making a stock investment, and the telephone number and residential address reserved by user C when making a bank card payment, the data are divided and classified according to the business categories corresponding to each data, and three data files can be obtained, namely: the data file corresponding to the current deposit business category, which includes the telephone number and residential address of user A, and the telephone number and residential address of user B; the data file corresponding to the stock investment business category, which includes the telephone number and residential address of user A; and the data file corresponding to the bank card payment business category, which includes the telephone number and residential address of user C.

[0042] It should also be noted that in the above example, the target data in the data file can also be stored in the data file after preprocessing such as data cleaning, data format conversion, feature extraction, data labeling, and data integration.

[0043] In another exemplary embodiment, assuming that the bank head office wants to carry out promotional activities for financial products under different business categories, taking one business category as an example, promotional information for the financial product is added to the bank's business application system to ensure that users can see it and click to enter the product details interface. When the user clicks on the promotional information and enters the product details interface, the system will record the user's relevant information (for example, the user's mobile phone number, name, age, occupation, residential address, etc.), that is, the above-mentioned target data, and store the target data in the business database for bank staff to retrieve. After relevant data processing, a data file is generated in the production environment. The data file may include but is not limited to:

[0044] S1, user log: records the user's behavior of clicking on promotional information, including click time, user ID, click source, etc.

[0045] S2, user information: including user's mobile phone number, name, age, occupation, residential address, etc.;

[0046] S3, product details access record: records detailed information about users visiting the product details interface, such as access time, duration of stay, page browsing path, etc.

[0047] S4, user feedback data file: If the user provides feedback on the product details interface, this feedback information will also be recorded;

[0048] S5, transaction data file: If the user makes a purchase or other transaction after viewing the product details, the relevant transaction data will also be recorded;

[0049] S6, user interaction data file: records the user's interaction with the product details interface, such as clicking links, filling out forms, etc.

[0050] S7, user portrait data file: User portraits built based on user behavior and personal information, used for more accurate marketing.

[0051] Next, the data file is sent to the bank head office business platform in the application environment for display, so that the bank head office business platform can assign work tasks related to the financial product to each bank branch business platform based on the target data recorded in the data file. Each bank branch business platform will execute corresponding promotion activities, for example, promote the financial product through the branch's channels, and assign corresponding promotion tasks according to the location area of ​​the branch corresponding to each data in the target data.

[0052] Furthermore, in the process of sending data files to the bank head office business platform in the application environment, it may include but is not limited to building a NAS service based on the ETL server. It can be understood that, assuming that the business personnel log in to the data management system through the data sandbox, the data management system initiates a data transmission request to the intelligent management system. Here, the intelligent management system is used to build a NAS service with the ETL server. The data files are determined in the intelligent management system, including but not limited to:

[0053] S1, directly upload the data file in the data management system;

[0054] S2, the intelligent management system determines the target data to be obtained according to the request parameters in the data transmission request, then obtains the target data from the business data, and then the intelligent management system generates a data file according to the business category of the target data;

[0055] In an exemplary embodiment, if the data file is determined through step S1, the data file can be stored in a network shared directory, and the above-mentioned file generation script deployed on the ETL server is periodically run to determine whether the above-mentioned data file exists in the above-mentioned network shared directory. If the above-mentioned data file exists in the network shared directory, the list file corresponding to the data file is determined according to the data attributes of the target data, including but not limited to determining the list file by establishing an index of the data file, file naming rules, metadata storage, etc.; for example, use a task scheduling tool (such as cron job, Windows Task Scheduler, etc.) to set up periodic tasks to ensure that the file generation script can access the network shared directory and generate a list file.

[0056] In another exemplary embodiment, if the data file is determined through step S2, then after the intelligent management system generates the data file according to the business category of the target data, you can choose to directly run the above-mentioned file generation script to determine the list file, or also periodically run the above-mentioned file generation script deployed on the ETL server to determine whether the above-mentioned data file exists in the above-mentioned network shared directory. If the above-mentioned data file exists in the network shared directory, then determine the list file corresponding to the data file according to the data attributes of the target data, for example, run the file generation script regularly at 22:00 every day.

[0057] S206, when a list file is detected in a network shared directory, the data file is transmitted to the business platform at a preset time according to the list file, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain target data.

[0058] Optionally, in the implementation of this application, the above-mentioned business platform may include but is not limited to being used to display the above-mentioned data files and perform further data processing on the data files. The business platform is deployed in the application environment and can be directly accessed through the network, while the data management system is deployed in the production environment and cannot be directly accessed through the network. The ETL server (target server) can be used to transfer files between the two through NFT.

[0059] Exemplarily, it may include but is not limited to deploying the above-mentioned file detection script on the ETL server, running the file detection script at the above-mentioned preset time, and once a list file is detected in the network shared directory, transferring the data file indicated by the list file to the business platform. For example, the preset time is 24:00 every day.

[0060] In an exemplary embodiment, taking the banking system business processing application scenario as an example, it includes a data sandbox, an integrated data management system, an intelligent management system, and an ETL server in the production environment, wherein the intelligent management system and the ETL server share a NAS storage directory (the above-mentioned network shared directory), and the ETL server provides NAS services); banking applications and a head office platform (business platform) in the application environment; its purpose is that the head office platform needs to obtain the target data pre-stored in the business database.

[0061] It should be noted that the target data source is the target data uploaded by users in application environments such as banking applications, or the internal target data in the production environment (such as model parameters of the measurement model). The target data is securely transmitted from the production environment to the head office platform for display and for the head office platform to allocate tasks to branches.

[0062] First, the first upload method: After the employee processes the target data, he obtains the data file, logs in to the comprehensive data management system, and uploads the data file to the intelligent management system. The intelligent management system stores the data file in the NAS storage directory. Then the ETL server executes the above file generation script and regularly checks the NAS storage directory. If the data file exists, it generates the .lst parameter file (the above list file) based on the data file's file name, node number, date and other parameters. It is also stored in the NAS storage directory. The NAS storage directory here stores the data file and the .lst parameter file corresponding to the data file.

[0063] Secondly, the second upload method: employees log in to the integrated data management system, provide the data conditions (time, business type, etc.), node number and file name to be transmitted as parameters, and request the intelligent management system to receive the target data. The intelligent management system directly obtains the target data from the database and generates a data file and a .lst parameter file based on the target data. The NAS storage directory here stores the data file and the .lst parameter file corresponding to the data file;

[0064] Next, the ETL server regularly runs the above file detection script to scan the NAS storage directory. If the .lst parameter file exists, the data file corresponding to the data file directory in the .lst file is uploaded to the head office platform through the nft service.

[0065] Through the embodiment of the present application, a data transmission request initiated by a receiving data management system is adopted, wherein the data transmission request is used to request the transmission of target data; in response to the data transmission request, a list file is detected in the network shared directory, wherein the list file is used to indicate the data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file; in the case where the list file is detected in the network shared directory, the data file is transmitted to the business platform at a preset time according to the list file, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit the initiation of a request to obtain the target data to the data management system, and the purpose of improving data processing is achieved by automating the steps of transmission and verification of the target data. At the same time, the security of the data transmission process is effectively guaranteed, thereby achieving the technical effect of saving data transmission time and simplifying the data processing process, thereby solving the technical problem of low data processing efficiency.

[0066] As an optional solution, when the above-mentioned list file is detected in the above-mentioned network shared directory, before the above-mentioned data file is transmitted to the business platform according to the above-mentioned list file, the above-mentioned method also includes: receiving the above-mentioned data file uploaded in the above-mentioned data management system through a secure file transfer protocol, and storing the above-mentioned data file in the above-mentioned network shared directory; establishing an index of the above-mentioned data file according to the data attributes of the above-mentioned target data, and generating the above-mentioned list file; storing the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0067] Optionally, in an embodiment of the present application, the above-mentioned transmission identifier may include but is not limited to characters, numbers, symbols, etc., and the transmission identifier may be determined based on the type, size, source or other business rules of the file. The index of the above-mentioned data file is used to quickly locate and retrieve data files, and the above-mentioned secure file transfer protocol includes but is not limited to FTPS (FTP Secure), SFTP (SSH File Transfer Protocol) or SCP (Secure Copy Protocol), etc.

[0068] For example, the data file can be quickly found through the index, and the transfer identifier can be used to decide whether to transfer the data file to the service platform.

[0069] In an exemplary embodiment, the SFTP security protocol is used to ensure the security of data files during transmission. The intelligent management system stores the received data files in a network shared directory, which can be a folder on the target server and can be accessed by multiple users or systems.

[0070] Next, an index is established for the data file based on the data attributes of the target data (such as file name, size, type, creation date, etc.), and a list file containing the data file index and transfer identifier is created. The list file is also stored in a network shared directory so that other systems or users can access it. Through the index recorded in the list file, the system can quickly locate the storage path of the data file, thereby improving retrieval efficiency; and, based on the transfer identifier recorded in the list file, determine whether to transfer the data file to the business platform. For example, if the transfer identifier is set to "yes", the file will be transferred; if it is set to "no", the file will be retained in the network shared directory.

[0071] As an optional solution, when the above-mentioned list file is detected in the above-mentioned network shared directory, before the above-mentioned data file is transmitted to the business platform according to the above-mentioned list file, the above-mentioned method also includes: determining target parameters related to the target data in response to the target interaction action occurring in the above-mentioned data management system, wherein the above-mentioned target parameters are used to indicate the data attributes of the above-mentioned target data, and the above-mentioned target interaction operation is used to indicate the acquisition of the above-mentioned target data from the business database; initiating a data acquisition request to the business database according to the target parameters in the above-mentioned data transmission request, and acquiring the above-mentioned target data in the above-mentioned business database; generating the above-mentioned data file and the above-mentioned list file according to the above-mentioned target data and the above-mentioned target parameters.

[0072] Optionally, in an embodiment of the present application, the above-mentioned target interaction action can be understood as, after the user logs in to the data management system, creating a request on the system interface, indicating whether the target data in the detection business database is safe and can be accessed, that is, whether the target data is in the above-mentioned target state; the above-mentioned data attributes include but are not limited to the data conditions required for transmission (time, business type, etc.), the nft transmission node number (the above-mentioned transmission node) and the file name corresponding to the target data, etc.

[0073] In an exemplary embodiment, assuming that the target data is sales data within a specific time period, the user first logs in to the data management system, creates a request on the system interface, and specifies the conditions for the data they want to obtain (for example, sales data for January 2023). The data management system recognizes the user's request and determines the target parameters based on the provided conditions. It then encapsulates a data transmission request based on the target parameters and sends it to the intelligent management system. The intelligent management system parses the data transmission request, obtains the target parameters, and then generates a data acquisition request according to the target parameters. The intelligent management system initiates the data acquisition request to the business database. After obtaining the target data returned by the business database, it generates a data file and a list file based on the target parameters.

[0074] Through the above embodiments, the purpose of improving data management and transmission efficiency is achieved, and the technical effects of ensuring data security and improving data processing speed are achieved.

[0075] As an optional solution, the above-mentioned data file and the above-mentioned list data are generated according to the above-mentioned target data and the above-mentioned target parameters, including: dividing the above-mentioned target data according to the business category of the above-mentioned target data to obtain the above-mentioned data file; establishing an index of the above-mentioned data file according to the above-mentioned target parameters, generating the above-mentioned list file, and storing the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0076] For example, the target data is the telephone number and residential address reserved by user A when making a current deposit, the telephone number and residential address reserved by user B when making a current deposit, the telephone number and residential address reserved by user A when making a stock investment, and the telephone number and residential address reserved by user C when making a bank card payment. Then, the data are divided and classified according to the business categories corresponding to each data, and three data files can be obtained, namely: the data file corresponding to the current deposit business category, which includes the telephone number and residential address of user A, and the telephone number and residential address of user B; the data file corresponding to the stock investment business category, which includes the telephone number and residential address of user A; and the data file corresponding to the bank card payment business category, which includes the telephone number and residential address of user C.

[0077] Furthermore, after obtaining the data file, you can follow the steps below:

[0078] S1, index creation: Create an index for each data file, which contains the storage path of the data file;

[0079] S2, Generate List File: Create a list file containing the indexes of all data files and their corresponding transfer identifiers. The transfer identifier can be a simple flag, such as "yes" or "no", indicating whether the data file needs to be transferred to the business platform.

[0080] S3, storage list file: store the list file in a network shared directory;

[0081] S4, data file transmission: according to the transmission identifier in the list file, the data file to be transmitted is sent to the business platform;

[0082] S5, update index and list files: If new data files are generated, or the status of existing data files may change (for example, a data file that originally did not need to be transferred now needs to be transferred), the list file can be updated regularly.

[0083] Through the embodiments of the present application, a list file is used to transmit specified data files, thereby achieving the technical effect of improving the transmission accuracy of data files.

[0084] As an optional solution, the above method also includes at least one of the following: obtaining a list number in the above data management system, wherein the above list number is used to indicate the upload time and upload type of the above data file; receiving the above data transmission request initiated by the above data management system based on the above list number and the transmission node, wherein the above transmission node is determined by the business category of the above target data; when the above target data is in a target state, receiving the above data transmission request initiated by the above data management system based on the above transmission node, wherein the above target state indicates that the above target data is allowed to be accessed.

[0085] Optionally, in an embodiment of the present application, the above-mentioned list number can be pre-created by business personnel and directly entered into the data management system for tracking and managing data files. It may include information such as data file upload time, uploader, file type, etc., or generated by the data management platform according to the data attributes of the target data. The above-mentioned transmission nodes include but are not limited to NFT nodes, etc.

[0086] In an exemplary embodiment, assuming that the data file is uploaded directly in the data management system, a data transmission request can be initiated to the intelligent management system through the list number and the transmission node; assuming that the data file requires the intelligent management system to request the target data in the business database and then generate the corresponding data file itself, then the data transmission request can be directly initiated to the intelligent management system through the transmission node without the list number.

[0087] As an optional solution, the above method also includes: periodically running a file generation script to determine whether the above data file exists in the above network shared directory, wherein the above file generation script is deployed on the target server, and the above target server allows access to the above network shared directory; when the above data file exists in the above network shared directory, establishing an index of the above data file according to the above target parameters, generating the above list file, and storing the above list file in the above network shared directory, wherein the index of the above data file is used to indicate the storage path of the above data file, and the above list file includes the index of the above data file and a transmission identifier, and the above transmission identifier is used to indicate whether the above data file needs to be transmitted to the above business platform.

[0088] As an optional solution, the above method also includes: periodically running a file detection script to detect whether the above list file exists in the above network shared directory, wherein the above file detection script is deployed on the target server, the above target server allows access to the above network shared directory, and the above target server is used to periodically transmit the above data files to the above business platform.

[0089] In an exemplary embodiment, the file detection script first detects the network shared directory and finds the presence of a list file therein, then reads the list file, obtains detailed information of the data file recorded in the list file (including but not limited to file index, file path, file generation time, file category, file size, etc.), and starts transferring the data file to the business platform.

[0090] It should be noted that the file detection script can also be set to only transmit data files that meet the preset file transfer conditions indicated by the list file. For example, the file transfer conditions indicate the transmission of data files within a specified time period, or the transmission of data files of a specified file size, etc. This application does not limit this.

[0091] Exemplarily, the above-mentioned file generation script is used to determine whether the above-mentioned data file exists in the above-mentioned network shared directory. If the above-mentioned data file exists in the network shared directory, the list file corresponding to the data file is determined according to the data attributes of the target data, including but not limited to determining the list file by establishing an index of the data file, file naming rules, metadata storage, etc.; for example, use a task scheduling tool (such as cron job, Windows Task Scheduler, etc.) to set periodic tasks to ensure that the file generation script can access the network shared directory and generate a list file.

[0092] For example, taking the application scenario of business processing of the banking system as an example, considering that in the existing technology, when transmitting sensitive data generated in the process of business processing of the banking system, it is often carried out through the existing auxiliary transmission system, and basically manual transmission of data files is required. There is a high probability of forgetting to destroy, transmission loopholes and other risks. In addition, the existing process system also processes repeatedly at a fixed frequency, requiring business and related technical personnel to waste manpower and time to cooperate throughout the process, and high-efficiency work cannot be achieved. Because the process of each link takes a long time, there is also a daily time delay in transmitting via NFT (network file transfer) to the head office promotion platform, and it is impossible to push the promotion data to the business personnel in a timely manner.

[0093] Specifically, the only way currently for production data to be pushed to the head office's promotion platform is through the ETL server in the production environment. However, the sensitive production data extracted by the business through the data distribution center can only be stored in the data sandbox. After this sensitive data is processed or confirmed by the business department, there is no direct and convenient channel to push the processed production data to the ETL server. It is only possible to create a work order through the data distribution center system in the data sandbox. The business personnel will transfer the processed files to the technical staff through the sandbox. The transmission process requires submission, review, download and use, and other process steps, which are implemented by technical staff with different responsibilities. After receiving the file, the transfer machine is used to manually transfer the file to the production ETL server, execute the file push script, and manually push the file to the head office's promotion platform.

[0094] Based on the above problems, this application realizes a data management system based on the collection and transmission of promotion sensitive data and its implementation method, which enables business personnel to automatically upload data on demand and transmit it to the promotion platform on time through tasks at preset times. This solution improves work efficiency, strengthens the timeliness of promotion, and reduces the waste of human resources and the risks that may be caused by human errors. Figure 3 This is a schematic diagram of an optional data processing method according to an embodiment of the present application, based on the data management system for promoting sensitive data collection and transmission such as Figure 3 As shown, including but not limited to:

[0095] S1. Business personnel log in to the employee channel through UASS (Ubiquitous Authentication and Security System) employee identity authentication, enter the promotion platform menu, and establish the corresponding external promotion list according to needs (for example, mobile phone numbers uploaded by users through the bank's application system, high-security data generated by the bank's internal business, such as model parameters of the business evaluation model, etc.), and record the list number.

[0096] S2, the business personnel first log in to the data sandbox, log in to the integrated data management system through UASS employee authentication, enter the file upload menu, and select the upload type according to the needs. The first upload method is to directly select the business category, such as installment payment business, enter the file upload page, enter the established list number, and select the corresponding NFT transmission node number (network file transmission node, the same type of business can use the same or different transmission nodes), upload the processed data file to the integrated data management system, and send an HTTP request to notify the intelligent management system (production system) of the relevant information of the application file. The application uses SFTP (Secure File Transfer Protocol) to transfer the data file to the integrated data management system. TransferProtocol)) and parameter information to extract data files and store them in NAS service storage (network attached storage). The first upload method requires that the script in the ETL server generate a .lst parameter file from the data file's file name, node number, date and other parameters (indexing operation) at a preset time and put it in the NAS service storage directory. Every day, the task on the ETL server scans the NAS service storage directory of the day in the NAS service storage at the preset time. If the .lst parameter file exists, the data file corresponding to the list directory in the .lst file (file suffix is ​​lst) is transmitted to the head office promotion platform through nft.

[0097] It should be noted that an ETL server is a server specifically used to perform the ETL (Extract, Transform, Load) process. ETL is a data processing method commonly used in data warehouses, data integration, and data migration. It includes the following three steps: Extraction: extracting data from various data sources (such as databases, file systems, APIs, etc.); Transformation: cleaning, converting, and processing the extracted data to meet the needs of the target system; Loading: loading the transformed data into the target system (such as a data warehouse, data lake, data mart, etc.). In the embodiment of the present application, the above-mentioned ETL server can deploy NAS services.

[0098] S3, the second upload method is that the data to be transmitted is generated by the application system on time according to the business logic, and the sensitive data is stored in the database. The data accuracy can be queried and confirmed according to the conditions on the comprehensive data management system, and then the corresponding NFT transmission node number is selected, and the upload application is submitted. The data conditions to be transmitted (time, business type, etc.), node number and file name, etc. are transmitted as parameters to the intelligent management system application through an HTTP request. The application then reads the data in the database according to the parameters and generates a data file, as well as the .lst parameter file required for NFT transmission (.lst file usually contains a file index to indicate which files to transfer) and stores them in the NAS service storage directory.

[0099] S4. After the application successfully stores the file in the NAS service storage, the file upload-related information (uploaded file path, file name, etc.), employee ID and log are updated or inserted into the database. The integrated data management system can request the business database through http and query the records uploaded by the user on the page to achieve a data preview effect.

[0100] S5. After T+2 (T is the upload time of sensitive data), business personnel can query the required sensitive data information in a pre-established external list (which can be established on the promotion platform of the employee channel) through the promotion platform of the employee channel.

[0101] Through the embodiments of the present application, the functions, nodes, list numbers and other features of file upload are automatically selectable, and can handle various push requirements and transmission requests of push platforms; and, an automatic file transfer process is realized between the data sandbox, intelligent management system, and NFT server, opening up the channel for file transmission between the business and the head office platform, and realizing a unified system process for uploading and downloading.

[0102] In addition, the embodiment of the present application has multiple processing methods for file upload, which can handle various types of data under different forms and different rules. It can be uploaded manually by business, or it can be automatically processed according to business logic and then uploaded after confirmation by the business. The transmission file can be automatically generated according to the rules, and the file can be automatically transmitted and destroyed at a preset time through a script, rather than manual operation of the entire process, avoiding the mistakes and risks that may be caused by manual operation, and solving the problem of data uploading to the bank head office platform process as the starting point, and establishing a system to form an automatic transmission file as the end point. Do not perform unnecessary operations, and do not need to manually handle the transmission process. Through the comprehensive data management system, the branch intelligent management system, and the nft transmission script, data can be automatically and securely pushed to the head office platform. The above steps are all performed automatically, avoiding mistakes and problems in human operations. The present invention can automatically transmit data files, greatly reducing the investment of human resources, and when business personnel have high requirements for the timeliness of data required for promotion or the head office and the demand is relatively frequent, it can be operated at any time, which is conducive to relevant personnel to quickly obtain data on the platform for business operations.

[0103] It is understandable that in the specific implementation of this application, sensitive data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0104] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0105] According to another aspect of the embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 4 As shown, the device includes:

[0106] The receiving module 402 is configured to receive a data transmission request initiated by the data management system, wherein the data transmission request is used to request transmission of target data;

[0107] a detection module 404 configured to detect, in response to the data transmission request, whether a list file exists in the network shared directory, wherein the list file is used to indicate data files to be transmitted, the data files including the target data, the data files being files uploaded by the data management system and / or files obtained by dividing the target data according to the service category of the target data, and the network shared directory being used to store the list file and the data files;

[0108] The transmission module 406 is used to transmit the data file to the business platform at a preset time according to the list file when the list file is detected in the network shared directory, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

[0109] As an optional solution, the above-mentioned device is also used to: when the above-mentioned list file is detected in the above-mentioned network shared directory, before transmitting the above-mentioned data file to the business platform according to the above-mentioned list file, receive the above-mentioned data file uploaded in the above-mentioned data management system through a secure file transfer protocol, and store the above-mentioned data file in the above-mentioned network shared directory; establish an index of the above-mentioned data file according to the data attributes of the above-mentioned target data, and generate the above-mentioned list file; store the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0110] As an optional solution, the above-mentioned device is also used to: when the above-mentioned list file is detected in the above-mentioned network shared directory, before transmitting the above-mentioned data file to the business platform according to the above-mentioned list file, determine the target parameters related to the target data in response to the target interaction action occurring in the above-mentioned data management system, wherein the above-mentioned target parameters are used to indicate the data attributes of the above-mentioned target data, and the above-mentioned target interaction operation is used to indicate the acquisition of the above-mentioned target data from the business database; initiate a data acquisition request to the business database according to the target parameters in the above-mentioned data transmission request, and acquire the above-mentioned target data in the above-mentioned business database; generate the above-mentioned data file and the above-mentioned list file according to the above-mentioned target data and the above-mentioned target parameters.

[0111] As an optional solution, the above-mentioned device is used to generate the above-mentioned data file and the above-mentioned list data according to the above-mentioned target data and the above-mentioned target parameters in the following manner: dividing the above-mentioned target data according to the business category of the above-mentioned target data to obtain the above-mentioned data file; establishing an index of the above-mentioned data file according to the above-mentioned target parameters, generating the above-mentioned list file, and storing the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0112] As an optional solution, the above-mentioned device is also used for at least one of the following: obtaining a list number in the above-mentioned data management system, wherein the above-mentioned list number is used to indicate the upload time and upload type of the above-mentioned data file; receiving the above-mentioned data transmission request initiated by the above-mentioned data management system based on the above-mentioned list number and the transmission node, wherein the above-mentioned transmission node is determined by the business category of the above-mentioned target data; when the above-mentioned target data is in a target state, receiving the above-mentioned data transmission request initiated by the above-mentioned data management system based on the above-mentioned transmission node, wherein the above-mentioned target state indicates that the above-mentioned target data is allowed to be accessed.

[0113] As an optional solution, the above-mentioned device is also used to: periodically run a file generation script to determine whether the data file exists in the network shared directory, wherein the file generation script is deployed on the target server, and the above-mentioned target server allows access to the above-mentioned network shared directory; when the above-mentioned data file exists in the above-mentioned network shared directory, establish an index of the above-mentioned data file according to the above-mentioned target parameters, generate the above-mentioned list file, and store the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0114] As an optional solution, the above-mentioned device is also used to: periodically run a file detection script to detect whether the above-mentioned list file exists in the above-mentioned network shared directory, wherein the above-mentioned file detection script is deployed on the target server, the above-mentioned target server allows access to the above-mentioned network shared directory, and the above-mentioned target server is used to periodically transmit the above-mentioned data file to the above-mentioned business platform.

[0115] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0116] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0117] According to one aspect of the present application, a computer program product is provided, which includes a computer program.

[0118] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0119] Figure 5 The block diagram schematically shows a computer system structure of an electronic device used to implement an embodiment of the present application.

[0120] It should be noted that Figure 5 The computer system 500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0121] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 into the random access memory (RAM) 503. Various programs and data required for system operation are also stored in the random access memory 503. The CPU 501, the read-only memory 502, and the random access memory 503 are connected to each other via a bus 504. An input / output interface 505 (i.e., an I / O interface) is also connected to the bus 504.

[0122] The following components are connected to the input / output interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a local area network card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that a computer program read therefrom can be installed into the storage section 508 as needed.

[0123] In particular, according to an embodiment of the present application, the processes described in the various method flow charts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 509 and / or installed from a removable medium 511. When the computer program is executed by the central processing unit 501, the various functions defined in the system of the present application are performed.

[0124] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit 501, various functions provided by the embodiments of the present application are performed.

[0125] According to another aspect of the embodiment of the present application, an electronic device for implementing the above data processing method is also provided. The electronic device may be Figure 1 The terminal device or server shown in FIG. This embodiment is described by taking the electronic device as a terminal device as an example. Figure 6 As shown, the electronic device includes a memory 602 and a processor 604. The memory 602 stores a computer program, and the processor 604 is configured to execute the steps in any of the above method embodiments through the computer program.

[0126] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0127] Optionally, in this embodiment, the above-mentioned processor can be configured to execute the methods in each embodiment of the present application through a computer program.

[0128] Alternatively, those skilled in the art will appreciate that Figure 6 The structure shown is for illustration only. Figure 6 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 6 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 6 Different configurations shown.

[0129] Among them, the memory 602 can be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method and device in the embodiments of the present application. The processor 604 executes various functional applications and data processing by running the software programs and modules stored in the memory 602, that is, realizing the above-mentioned data processing method. The memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 602 may further include a memory remotely located relative to the processor 604, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include but are not limited to the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 602 can be used specifically but not limited to store target data and other information. As an example, such as Figure 6 As shown, the memory 602 may include, but is not limited to, the receiving module 402, the detecting module 404, and the transmitting module 406 in the data processing device. In addition, it may also include, but is not limited to, other module units in the data processing device, which will not be described in detail in this example.

[0130] Optionally, the transmission device 606 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one embodiment, the transmission device 606 includes a network interface controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In one embodiment, the transmission device 606 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0131] In addition, the electronic device further includes: a display 608 for displaying the data file; and a connection bus 610 for connecting various module components in the electronic device.

[0132] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting multiple nodes via network communication. The nodes may form a peer-to-peer network, and any computing device, such as a server, terminal, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0133] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the data processing method provided in various optional implementations of the above-mentioned data processing aspects.

[0134] Optionally, in this embodiment, the above-mentioned computer-readable storage medium can be configured to store data for executing the methods in various embodiments of the present application.

[0135] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0136] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0137] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above-mentioned computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing one or more electronic devices to execute all or part of the steps of the method described in each embodiment of the present application.

[0138] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed applications can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0141] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0142] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: include: receiving a data transmission request initiated by a data management system, wherein the data transmission request is used to request transmission of target data; In response to the data transmission request, detecting whether a list file exists in a network shared directory, wherein the list file is used to indicate data files to be transmitted, the data files include the target data, the data files are files uploaded by the data management system, and / or files obtained by dividing the target data according to the service category of the target data, and the network shared directory is used to store the list file and the data files; When the list file is detected in the network shared directory, the data file is transmitted to the business platform at a preset time according to the list file, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

2. The method according to claim 1, characterized in that When the list file is detected in the network shared directory, before transmitting the data file to the service platform according to the list file, the method further includes: Receiving the data file uploaded in the data management system through a secure file transfer protocol, and storing the data file in the network shared directory; Establishing an index of the data file according to the data attributes of the target data, and generating the list file; The list file is stored in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

3. The method according to claim 1, characterized in that When the list file is detected in the network shared directory, before transmitting the data file to the service platform according to the list file, the method further includes: In response to a target interaction action occurring in the data management system, determining a target parameter related to the target data, wherein the target parameter is used to indicate a data attribute of the target data, and the target interaction action is used to instruct to obtain the target data from a business database; Initiate a data acquisition request to a business database according to the target parameters in the data transmission request, and acquire the target data in the business database; The data file and the list file are generated according to the target data and the target parameters.

4. The method according to claim 3, characterized in that The step of generating the data file and the list file according to the target data and the target parameters includes: dividing the target data according to the business category of the target data to obtain the data file; Establish an index of the data file according to the target parameters, generate the list file, and store the list file in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

5. The method according to claim 1, wherein The method further comprises at least one of the following: Obtaining a list number in the data management system, wherein the list number is used to indicate the upload time and upload type of the data file; receiving the data transmission request initiated by the data management system based on the list number and a transmission node, wherein the transmission node is determined by the service category of the target data; In a case where the target data is in a target state, the data transmission request initiated by the data management system based on the transmission node is received, wherein the target state indicates that the target data is allowed to be accessed.

6. The method according to claim 3, characterized in that The method further comprises: Periodically running a file generation script to determine whether the data file exists in the network shared directory, wherein the file generation script is deployed on a target server, and the target server is allowed to access the network shared directory; When the data file exists in the network shared directory, an index of the data file is established according to the target parameter, the list file is generated, and the list file is stored in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

7. The method according to claim 1, characterized in that The method further comprises: Periodically run a file detection script to detect whether the list file exists in the network shared directory, wherein the file detection script is deployed on a target server, the target server allows access to the network shared directory, and the target server is used to periodically transmit the data file to the business platform.

8. A data processing device, characterized in that: include: A receiving module, configured to receive a data transmission request initiated by a data management system, wherein the data transmission request is used to request transmission of target data; a detection module, configured to detect, in response to the data transmission request, whether a list file exists in a network shared directory, wherein the list file is used to indicate data files to be transmitted, the data files include the target data, the data files are files uploaded by the data management system, and / or files obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data files; A transmission module is used to transmit the data file to the business platform at a preset time according to the list file when the list file is detected in the network shared directory, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein the computer program can be executed by an electronic device to perform the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images