Data processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a data processing method and device, a storage medium and electronic equipment. The method comprises the steps that a data transmission request initiated by a data management system is received, and the data transmission request is used for requesting transmission of target data; in response to the data transmission request, detecting whether a list file exists in a network shared directory, the list file being used for indicating a to-be-transmitted data file, the data file being a file directly uploaded by a data management system and / or a file obtained by dividing the target data according to a service category of the target data, and the list file being used for indicating the to-be-transmitted data file; the network sharing directory is used for storing list files and data files; and under the condition that the list file is detected in the network shared directory, transmitting the data file to the service platform at a preset moment according to the list file. The technical problem that the data processing efficiency is low is solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a data processing method and device, a storage medium, and an electronic device. Background Art

[0002] At present, in the relevant technologies, system work orders are mainly manually established, and the process approval of each system link is carried out in sequence, so as to achieve the safe transmission of part of the data in the production environment to the application environment for use by the business platform in the application environment. In this process, due to the long time required for the process of each link and the untimely data transmission, the technical problem of low data processing efficiency occurs.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a data processing method and device, a storage medium, and an electronic device to at least solve the technical problem of low data processing efficiency.

[0005] According to one aspect of an embodiment of the present application, a data processing method is provided, comprising: receiving a data transmission request initiated by a data management system, wherein the data transmission request is used to request the transmission of target data; in response to the data transmission request, detecting whether there is a list file in a network shared directory, wherein the list file is used to indicate data files to be transmitted, the data files include the target data, the data files are files uploaded by the data management system, and / or files obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data files; in a case where the list file is detected in the network shared directory, transmitting the data files to a business platform at a preset time according to the list file, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

[0006] According to another aspect of an embodiment of the present application, a data processing device is also provided, including: a receiving module, used to receive a data transmission request initiated by a data management system, wherein the data transmission request is used to request the transmission of target data; a detection module, used to detect whether there is a list file in the network shared directory in response to the data transmission request, wherein the list file is used to indicate the data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file; a transmission module, used to transmit the data file to a business platform at a preset time according to the list file when the list file is detected in the network shared directory, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

[0007] Optionally, the device is also used to: when the list file is detected in the network shared directory, before transmitting the data file to the business platform according to the list file, receive the data file uploaded in the data management system through a secure file transfer protocol, and store the data file in the network shared directory; establish an index of the data file according to the data attributes of the target data to generate the list file; store the list file in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

[0008] Optionally, the device is also used to: when the list file is detected in the network shared directory, before transmitting the data file to the business platform according to the list file, determine the target parameters related to the target data in response to the target interaction action occurring in the data management system, wherein the target parameters are used to indicate the data attributes of the target data, and the target interaction operation is used to indicate obtaining the target data from the business database; initiate a data acquisition request to the business database according to the target parameters in the data transmission request to obtain the target data in the business database; and generate the data file and the list file according to the target data and the target parameters.

[0009] Optionally, the device is used to generate the data file and the list data according to the target data and the target parameters in the following manner: divide the target data according to the business category of the target data to obtain the data file; establish an index of the data file according to the target parameters, generate the list file, and store the list file in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

[0010] Optionally, the device is also used for at least one of the following: obtaining a list number in the data management system, wherein the list number is used to indicate the upload time and upload type of the data file; receiving the data transmission request initiated by the data management system based on the list number and a transmission node, wherein the transmission node is determined by the business category of the target data; and receiving the data transmission request initiated by the data management system based on the transmission node when the target data is in a target state, wherein the target state indicates that the target data is allowed to be accessed.

[0011] Optionally, the device is also used to: periodically run a file generation script to determine whether the data file exists in the network shared directory, wherein the file generation script is deployed on a target server, and the target server allows access to the network shared directory; when the data file exists in the network shared directory, establish an index of the data file according to the target parameters, generate the list file, and store the list file in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

[0012] Optionally, the device is also used to: periodically run a file detection script to detect whether the list file exists in the network shared directory, wherein the file detection script is deployed on a target server, the target server allows access to the network shared directory, and the target server is used to periodically transmit the data file to the business platform.

[0013] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned data processing method when running.

[0014] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above data processing method.

[0015] According to another aspect of the embodiments of the present application, there is further provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the data processing method through the computer program.

[0016] In an embodiment of the present application, a data transmission request initiated by a data management system is received, wherein the data transmission request is used to request the transmission of target data; in response to the data transmission request, a list file is detected in the network shared directory, wherein the list file is used to indicate the data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file; in the case where the list file is detected in the network shared directory, the data file is transmitted to the business platform at a preset time according to the list file, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit the initiation of a request to the data management system to obtain the target data, and the purpose of improving data processing is achieved by automating the steps of transmission and verification of the target data, and at the same time, the security of the data transmission process is effectively guaranteed, thereby achieving the technical effect of saving data transmission time and simplifying the data processing process, thereby solving the technical problem of low data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a schematic diagram of an application environment of an optional data processing method according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of an optional data processing method according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of an optional data processing method according to an embodiment of the present application;

[0021] Figure 4is a schematic structural diagram of an optional data processing device according to an embodiment of the present application;

[0022] Figure 5 is a schematic diagram of the structure of an optional data processing product according to an embodiment of the present application;

[0023] Figure 6 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] The present application is described below in conjunction with embodiments:

[0027] According to one aspect of an embodiment of the present application, a data processing method is provided. Optionally, in this embodiment, the data processing method can be applied to Figure 1 In the hardware environment composed of the server 101 and the terminal device 103 shown in FIG. Figure 1As shown, the server 101 is connected to the terminal 103 via a network, and can be used to provide services for the terminal device or the application installed on the terminal device. The application 107 can be a video application, an instant messaging application, a browser application, an educational application, a game application, etc. A database 105 may be set up on the server or independently of the server to provide data storage services for the server 101, for example, a game data storage server. The above-mentioned network may include, but is not limited to, a wired network and a wireless network, wherein the wired network includes, for example, a local area network, a metropolitan area network, and a wide area network; the wireless network includes, for example, Bluetooth, WIFI, and other networks that implement wireless communication; the terminal device 103 may be a terminal configured with an application program, and may include, but is not limited to, at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop computer, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a mixed reality (MR) terminal, and other computer devices; the above-mentioned server may be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0028] Combination Figure 1 As shown, the above data processing method can be executed by an electronic device, which can be a terminal device or a server. The above data processing method can be implemented by the terminal device or the server respectively, or by the terminal device and the server together.

[0029] The above is only an example and is not specifically limited in this embodiment.

[0030] Optionally, as an optional implementation, as Figure 2 As shown, the above data processing method includes:

[0031] S202, receiving a data transmission request initiated by a data management system, wherein the data transmission request is used to request transmission of target data;

[0032] Optionally, in an embodiment of the present application, the above-mentioned target data may include but is not limited to being input by the user in the business application system through a mouse and keyboard, touch screen, voice, etc., and then stored in the business database by the business application system to process relevant business data; or, the target data may also be some sensitive data generated in the production environment corresponding to the business application system, which can also be stored in the business database.

[0033] It can be understood that the business application system here can refer to different systems in different application environments. For example, taking banking business as an example, it can be a banking business application or banking business web system on a smart terminal, or it can be a bank’s internal application development system, etc. The smart terminals here can include but are not limited to mobile phones, computers, notebooks, smart wearable devices, etc.

[0034] In an exemplary embodiment, a user logs into a banking application using a mobile phone and enters an account number and password on the login interface. Assuming that the user logs in using a mobile phone number and a one-time verification code, the mobile phone number entered by the user is the above-mentioned target data.

[0035] In yet another exemplary embodiment, the business development department of a bank trains a business prediction model in the bank's internal application development system, and the model parameters corresponding to the model are the above-mentioned target data.

[0036] Furthermore, the above-mentioned data management system is also deployed in an internal production environment and isolated from the external application environment. Users can verify their identities through data sandboxes and other methods, log in to the data management system and choose to upload data files generated after processing the target data, or initiate a request in the data management system to instruct another intelligent management system in the production environment to obtain the target data stored in the business database, and then the intelligent management system processes the target data and generates a data file.

[0037] Furthermore, the above-mentioned data transmission request refers to a request for requesting the transmission of target data, which may include but is not limited to https requests, etc., and its request parameters include those for indicating the source of the target data, transmission address, data type, data storage location, data transmission protocol, etc.

[0038] S204, in response to the data transmission request, detecting whether there is a list file in the network shared directory, wherein the list file is used to indicate the data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file;

[0039] Optionally, in an embodiment of the present application, the above-mentioned data files can be files obtained by dividing and classifying according to the business categories corresponding to each data in the above-mentioned target data, and each data file contains data of the corresponding business field; or, divided according to the data type corresponding to each data in the target data, such as text data files, image data files, audio data files, etc.; or, divided according to the data source corresponding to each data in the target data, such as data files generated by business department A, data files generated by business department B, etc.; or, divided according to the data usage corresponding to each data in the target data, such as data files for analysis, data files for reporting, data files for backup, etc.; or, divided according to the data format corresponding to each data in the target data, such as CSV files, XML files, JSON files, database files, etc.

[0040] In an exemplary embodiment, it is assumed that the above-mentioned data file is a file obtained by dividing and classifying according to the business category corresponding to each data in the above-mentioned target data. Taking the banking business system as an example, the business categories include but are not limited to deposit business, loan business, investment business, payment business, etc. Specifically, it can be further subdivided into primary business, secondary business belonging to the sub-business of the primary business, etc., and divided and classified according to the sub-business type to which each data belongs. For example, the primary business is deposit business, loan business, investment business, and payment business. The secondary business under the deposit business is current deposits, time deposits, notice deposits, foreign currency deposits, savings deposits, etc.; the secondary business under the loan business is personal loans, corporate loans, credit card loans, mortgage loans, credit loans, etc.; the secondary business under the investment business is stock investment, fund investment, futures investment, foreign exchange investment, etc.; the secondary business under the payment business is bank card payment, electronic wallet payment, mobile payment, etc.

[0041] Furthermore, taking the target data as the telephone number and residential address reserved by user A when making a current deposit, the telephone number and residential address reserved by user B when making a current deposit, the telephone number and residential address reserved by user A when making a stock investment, and the telephone number and residential address reserved by user C when making a bank card payment, the data are divided and classified according to the business categories corresponding to each data, and three data files can be obtained, namely: a data file corresponding to the current deposit business category, which includes the telephone number and residential address of user A, and the telephone number and residential address of user B; a data file corresponding to the stock investment business category, which includes the telephone number and residential address of user A; and a data file corresponding to the bank card payment business category, which includes the telephone number and residential address of user C.

[0042] It should also be noted that in the above example, the target data in the data file can also be stored in the data file after preprocessing such as data cleaning, data format conversion, feature extraction, data labeling, and data integration.

[0043] In another exemplary embodiment, assuming that the head office of a bank wants to carry out promotion activities for financial products under different business categories, taking one of the business categories as an example, the promotional information of the financial product is added to the bank's business application system to ensure that the user can see and click to enter the product details interface. When the user clicks on the promotional information and enters the product details interface, the system will record the user's relevant information (for example, the user's mobile phone number, name, age, occupation, residential address, etc.), that is, the above-mentioned target data, and store the target data in the business database for bank staff to extract, and after relevant data processing, generate a data file in the production environment. The data file may include but is not limited to:

[0044] S1, user log: records the user's behavior of clicking on promotional information, including click time, user ID, click source, etc.;

[0045] S2, user information: including user's mobile phone number, name, age, occupation, residential address, etc.;

[0046] S3, product details access record: records detailed information about users’ access to the product details interface, such as access time, stay time, page browsing path, etc.

[0047] S4, user feedback data file: If the user provides feedback on the product details interface, the feedback information will also be recorded;

[0048] S5, transaction data file: If the user makes a purchase or other transaction after viewing the product details, the relevant transaction data will also be recorded;

[0049] S6, user interaction data file: records the user's interaction with the product details interface, such as clicking links, filling out forms, etc.;

[0050] S7, user portrait data file: user portrait built based on user behavior and personal information, used for more accurate marketing.

[0051] Next, the data file is sent to the bank head office business platform in the application environment for display, so that the bank head office business platform can assign work tasks related to the financial product to each bank branch business platform according to the target data recorded in the data file, and each bank branch business platform will execute corresponding promotion activities. For example, promote the financial product through the branch’s channels, assign corresponding promotion tasks according to the location area of ​​the branch corresponding to each data in the target data, etc.

[0052] Furthermore, in the process of sending the data file to the bank head office business platform in the application environment, it may include but is not limited to building a NAS service based on the ETL server. It can be understood that, assuming that the business personnel log in to the data management system through the data sandbox, the data management system initiates a data transmission request to the intelligent management system. The intelligent management system here is used to build a NAS service with the ETL server, and determine the data file in the intelligent management system, including but not limited to:

[0053] S1, directly upload data files in the data management system;

[0054] S2, the intelligent management system determines the target data to be obtained according to the request parameters in the data transmission request, then obtains the target data from the business data, and then the intelligent management system generates a data file according to the business category of the target data;

[0055] In an exemplary embodiment, if the data file is determined through step S1, the data file can be stored in a network shared directory, and the above-mentioned file generation script deployed on the ETL server is periodically run to determine whether the above-mentioned data file exists in the above-mentioned network shared directory. If the above-mentioned data file exists in the network shared directory, the list file corresponding to the data file is determined according to the data attributes of the target data, including but not limited to determining the list file by establishing an index of the data file, file naming rules, metadata storage, etc.; for example, use a task scheduling tool (such as cron job, Windows Task Scheduler, etc.) to set periodic tasks to ensure that the file generation script can access the network shared directory and generate a list file.

[0056] In another exemplary embodiment, if the data file is determined through step S2, then after the intelligent management system generates the data file according to the business category of the target data, you can choose to directly run the above-mentioned file generation script to determine the list file, or also periodically run the above-mentioned file generation script deployed on the ETL server to determine whether the above-mentioned data file exists in the above-mentioned network shared directory. If the above-mentioned data file exists in the network shared directory, then determine the list file corresponding to the data file according to the data attributes of the target data, for example, run the file generation script regularly at 22:00 every day.

[0057] S206, when a list file is detected in a network shared directory, the data file is transmitted to the business platform at a preset time according to the list file, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain target data.

[0058] Optionally, in the implementation of the present application, the above-mentioned business platform may include but is not limited to being used to display the above-mentioned data files and perform further data processing on the data files. The business platform is deployed in the application environment and can be directly accessed through the network, while the data management system is deployed in the production environment and cannot be directly accessed through the network. The ETL server (target server) can be used to transfer files between the two through NFT.

[0059] Exemplarily, it may include but is not limited to deploying the above-mentioned file detection script on the ETL server, running the file detection script at the above-mentioned preset time, and once the list file is detected in the network shared directory, the data file indicated by the list file is transmitted to the business platform. For example, the preset time is 24:00 every day.

[0060] In an exemplary embodiment, taking the banking system business processing application scenario as an example, it includes a data sandbox, a comprehensive data management system, an intelligent management system, and an ETL server in a production environment, wherein the intelligent management system and the ETL server share a NAS storage directory (the above-mentioned network shared directory), and the ETL server provides NAS services); banking applications and a head office platform (business platform) in the application environment; its purpose is that the head office platform needs to obtain target data pre-stored in the business database.

[0061] It should be noted that the target data source is the target data uploaded by users in application environments such as banking applications, or the internal target data in the production environment (model parameters of the measurement model, etc.). The target data is securely transmitted from the production environment to the head office platform for display, and for the head office platform to allocate tasks to branches;

[0062] First, the first upload method: After the employee processes the target data, he obtains the data file, logs in to the comprehensive data management system and uploads the data file to the intelligent management system, which stores the data file in the NAS storage directory. The ETL server then executes the above file generation script and periodically checks the NAS storage directory. If the data file exists, the .lst parameter file (the above list file) is generated based on the file name, node number, date and other parameters of the data file, and is still stored in the NAS storage directory. The NAS storage directory here stores the data file and the .lst parameter file corresponding to the data file.

[0063] Secondly, the second upload method: the employee logs in to the integrated data management system, takes the data conditions (time, business type, etc.), node number and file name to be transmitted as parameters, and requests the intelligent management system to receive the target data. The intelligent management system directly obtains the target data from the database, and generates data files and .lst parameter files according to the target data, and stores them in the NAS storage directory. The NAS storage directory here stores the data files and the .lst parameter files corresponding to the data files;

[0064] Next, the ETL server periodically runs the above file detection script to scan the NAS storage directory. If there is a .lst parameter file, the data file corresponding to the data file directory in the .lst file is uploaded to the head office platform through the nft service.

[0065] Through the embodiment of the present application, a data transmission request initiated by a data management system is received, wherein the data transmission request is used to request the transmission of target data; in response to the data transmission request, it is detected whether there is a list file in the network shared directory, wherein the list file is used to indicate the data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file; in the case where the list file is detected in the network shared directory, the data file is transmitted to the business platform at a preset time according to the list file, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit the initiation of a request to the data management system to obtain the target data, and the purpose of improving data processing is achieved by automating the steps of transmission and verification of the target data, and at the same time, the security of the data transmission process is effectively guaranteed, thereby achieving the technical effect of saving data transmission time and simplifying the data processing process, thereby solving the technical problem of low data processing efficiency.

[0066] As an optional scheme, in the case where the above-mentioned list file is detected in the above-mentioned network shared directory, before the above-mentioned data file is transmitted to the business platform according to the above-mentioned list file, the above-mentioned method also includes: receiving the above-mentioned data file uploaded in the above-mentioned data management system through a secure file transfer protocol, and storing the above-mentioned data file in the above-mentioned network shared directory; establishing an index of the above-mentioned data file according to the data attributes of the above-mentioned target data, and generating the above-mentioned list file; storing the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0067] Optionally, in an embodiment of the present application, the above-mentioned transmission identifier may include but is not limited to characters, numbers, symbols, etc., and the transmission identifier may be determined based on the type, size, source or other business rules of the file. The index of the above-mentioned data file is used to quickly locate and retrieve data files. The above-mentioned secure file transfer protocol includes but is not limited to FTPS (FTP Secure), SFTP (SSH File Transfer Protocol) or SCP (Secure Copy Protocol), etc.

[0068] Exemplarily, the data file can be quickly found through the index, and the transfer identifier can be used to decide whether to transfer the data file to the service platform.

[0069] In an exemplary embodiment, the SFTP security protocol is used to ensure the security of data files during transmission. The intelligent management system stores the received data files in a network shared directory, which can be a folder on the target server and can be accessed by multiple users or systems.

[0070] Next, an index is established for the data file according to the data attributes of the target data (such as file name, size, type, creation date, etc.), and a list file containing the data file index and transfer identifier is created. The list file is also stored in the network shared directory so that other systems or users can access it. Through the index recorded in the list file, the system can quickly locate the storage path of the data file, thereby improving the retrieval efficiency; and, based on the transfer identifier recorded in the list file, determine whether to transfer the data file to the business platform. For example, if the transfer identifier is set to "yes", the file will be transferred; if it is set to "no", the file will be retained in the network shared directory.

[0071] As an optional scheme, when the above-mentioned list file is detected in the above-mentioned network shared directory, before the above-mentioned data file is transmitted to the business platform according to the above-mentioned list file, the above-mentioned method also includes: determining target parameters related to the target data in response to the target interaction action occurring in the above-mentioned data management system, wherein the above-mentioned target parameters are used to indicate the data attributes of the above-mentioned target data, and the above-mentioned target interaction operation is used to indicate obtaining the above-mentioned target data from the business database; initiating a data acquisition request to the business database according to the target parameters in the above-mentioned data transmission request, and obtaining the above-mentioned target data in the above-mentioned business database; and generating the above-mentioned data file and the above-mentioned list file according to the above-mentioned target data and the above-mentioned target parameters.

[0072] Optionally, in an embodiment of the present application, the above-mentioned target interaction action can be understood as, after the user logs into the data management system, creating a request on the system interface, indicating whether the target data in the detection business database is safe and can be accessed, that is, whether the target data is in the above-mentioned target state; the above-mentioned data attributes include but are not limited to the data conditions required for transmission (time, business type, etc.), the nft transmission node number (the above-mentioned transmission node) and the file name corresponding to the target data, etc.

[0073] In an exemplary embodiment, assuming that the target data is sales data within a specific time period, the user first logs in to the data management system, creates a request on the system interface, and specifies the conditions for the data they want to obtain (for example, sales data for January 2023). The data management system recognizes the user's request and determines the target parameters based on the provided conditions. It then encapsulates a data transmission request based on the target parameters and sends it to the intelligent management system. The intelligent management system parses the data transmission request, obtains the target parameters therein, and then generates a data acquisition request according to the target parameters. The intelligent management system initiates the data acquisition request to the business database, and after obtaining the target data returned by the business database, generates a data file and a list file based on the target parameters.

[0074] Through the above embodiments, the purpose of improving data management and transmission efficiency is achieved, and the technical effects of ensuring data security and improving data processing speed are achieved.

[0075] As an optional solution, the above-mentioned data file and the above-mentioned list data are generated according to the above-mentioned target data and the above-mentioned target parameters, including: dividing the above-mentioned target data according to the business category of the above-mentioned target data to obtain the above-mentioned data file; establishing an index of the above-mentioned data file according to the above-mentioned target parameters, generating the above-mentioned list file, and storing the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0076] Exemplarily, the target data is the telephone number and residential address reserved by user A when making a current deposit, the telephone number and residential address reserved by user B when making a current deposit, the telephone number and residential address reserved by user A when making a stock investment, and the telephone number and residential address reserved by user C when making a bank card payment. Then, the data is divided and classified according to the business categories corresponding to each data, and three data files can be obtained, namely: a data file corresponding to the current deposit business category, which includes the telephone number and residential address of user A, and the telephone number and residential address of user B; a data file corresponding to the stock investment business category, which includes the telephone number and residential address of user A; and a data file corresponding to the bank card payment business category, which includes the telephone number and residential address of user C.

[0077] Furthermore, after obtaining the data file, you can follow the steps below:

[0078] S1, index creation: create an index for each data file, which contains the storage path of the data file;

[0079] S2, generate a list file: create a list file, which contains the indexes of all data files and the corresponding transmission identifiers. The transmission identifier can be a simple mark, such as "yes" or "no", to indicate whether the data file needs to be transmitted to the business platform;

[0080] S3, storage list file: store the list file in a network shared directory;

[0081] S4, data file transmission: according to the transmission identifier in the list file, the data file to be transmitted is sent to the business platform;

[0082] S5, update index and list files: if new data files are generated, or the status of existing data files may change (for example, data files that originally did not need to be transferred now need to be transferred), the list files may be updated regularly.

[0083] Through the embodiments of the present application, a list file is used to transmit a specified data file, thereby achieving the technical effect of improving the transmission accuracy of the data file.

[0084] As an optional scheme, the above method also includes at least one of the following: obtaining the list number in the above data management system, wherein the above list number is used to indicate the upload time and upload type of the above data file; receiving the above data transmission request initiated by the above data management system based on the above list number and the transmission node, wherein the above transmission node is determined by the business category of the above target data; when the above target data is in a target state, receiving the above data transmission request initiated by the above data management system based on the above transmission node, wherein the above target state indicates that the above target data is allowed to be accessed.

[0085] Optionally, in an embodiment of the present application, the above-mentioned list number can be created in advance by business personnel and directly entered into the data management system for tracking and managing data files, and may include information such as data file upload time, uploader, file type, etc., or generated by the data management platform according to the data attributes of the target data. The above-mentioned transmission nodes include but are not limited to nft nodes, etc.

[0086] In an exemplary embodiment, assuming that the data file is uploaded directly in the data management system, a data transmission request can be initiated to the intelligent management system through the list number and the transmission node; assuming that the data file requires the intelligent management system to request the target data in the business database and then generate the corresponding data file itself, then there is no need for a list number and a data transmission request can be directly initiated to the intelligent management system through the transmission node.

[0087] As an optional solution, the method further includes: periodically running a file generation script to determine whether the data file exists in the network shared directory, wherein the file generation script is deployed on a target server, and the target server allows access to the network shared directory; when the data file exists in the network shared directory, establishing an index of the data file according to the target parameters, generating the list file, and storing the list file in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

[0088] As an optional solution, the above method also includes: periodically running a file detection script to detect whether the above list file exists in the above network shared directory, wherein the above file detection script is deployed on the target server, the above target server allows access to the above network shared directory, and the above target server is used to periodically transmit the above data files to the above business platform.

[0089] In an exemplary embodiment, the file detection script first detects the network shared directory and finds the presence of a list file therein, then reads the list file, obtains detailed information of the data file recorded in the list file (including but not limited to file index, file path, file generation time, file category, file size, etc.), and starts transferring the data file to the business platform.

[0090] It should be noted that the file detection script can also be set to only transmit data files that meet preset file transfer conditions as indicated by the list file. For example, the file transfer conditions indicate the transmission of data files within a specified time period, or the transmission of data files of a specified file size, etc. This application does not limit this.

[0091] Exemplarily, the above-mentioned file generation script is used to determine whether the above-mentioned data file exists in the above-mentioned network shared directory. If the above-mentioned data file exists in the network shared directory, the list file corresponding to the data file is determined according to the data attributes of the target data, including but not limited to determining the list file by establishing an index of the data file, file naming rules, metadata storage, etc.; for example, use a task scheduling tool (such as cron job, Windows Task Scheduler, etc.) to set periodic tasks to ensure that the file generation script can access the network shared directory and generate a list file.

[0092] For example, taking the application scenario of business processing in the banking system as an example, considering that in the prior art, when transmitting sensitive data generated in the process of business processing in the banking system, it is often carried out through the existing auxiliary transmission system, and basically data files need to be transmitted manually, there is a high probability of forgetting to destroy, transmission loopholes and other risks. In addition, the existing process system also processes repeatedly at a fixed frequency, requiring business and related technical personnel to waste manpower and time to cooperate throughout the process, and high-efficiency work cannot be achieved. Because the time required for each link of the process is relatively long, there is also a daily time delay in transmitting via NFT (network file transfer) to the head office promotion platform, and it is impossible to push the promotion data to the business personnel in a timely manner.

[0093] Specifically, the only way for production data to be pushed to the head office promotion platform is through the ETL server in the production environment. However, the production sensitive data extracted by the business through the data distribution center can only be saved in the data sandbox. After the sensitive data is processed or confirmed by the business department, there is no direct and convenient channel to push the processed production data to the ETL server. The only way is to create a work order through the data distribution center system in the data sandbox. The business personnel transfer the processed files to the technical personnel through the sandbox. The transmission process requires submission, review, download and use, and other process steps, which are implemented by technical personnel with different responsibilities. After receiving the file, the transfer machine is used to manually transfer the file to the production ETL server, execute the file push script, and manually push the file to the head office promotion platform.

[0094] Based on the above problems, this application realizes a data management system based on the collection and transmission of promotion sensitive data and its implementation method, which realizes the solution of automatically uploading data by business personnel on demand and transmitting it to the promotion platform on time through tasks at preset times, thereby improving work efficiency, enhancing the timeliness of promotion, and reducing the waste of human resources and the risks that may be caused by human errors. Figure 3 is a schematic diagram of an optional data processing method according to an embodiment of the present application, based on promoting a data management system for sensitive data collection and transmission such as Figure 3 As shown, including but not limited to:

[0095] S1. The business personnel log in to the employee channel through UASS (Ubiquitous Authentication and Security System) employee identity authentication, enter the menu of the promotion platform, and establish the corresponding external promotion list according to the needs (for example, the mobile phone number uploaded by the user through the bank application system, the data with higher security generated by the bank's internal business, such as the model parameters of the business evaluation model, etc.), and record the list number.

[0096] S2, the business personnel first log in to the data sandbox, log in to the integrated data management system through UASS employee authentication, enter the file upload menu, and select the upload type according to the needs. The first upload method is to directly select the business category, such as installment payment business, enter the file upload page, enter the established list number, and select the corresponding nft transmission node number (network file transmission node, the same type of business can use the same or different transmission nodes), upload the processed data file to the integrated data management system, and send an http request to notify the intelligent management system (production system) of the relevant information of the application file, which is transmitted by the application through sftp (Secure File Transfer Protocol (Secure File Transfer Protocol TransferProtocol)) and parameter information to extract data files and store them in NAS service storage (network attached storage). The first upload method requires that the script in the ETL server generates a .lst parameter file at a preset time from the file name, node number, date and other parameters of the data file (indexing operation) and puts it in the NAS service storage directory. Every day, the task on the ETL server scans the NAS service storage directory of the current date in the NAS service storage at the preset time. If there is a .lst parameter file, the data file corresponding to the list directory in the .lst file (file suffix is ​​lst) is transmitted to the head office promotion platform through nft.

[0097] It should be noted that the ETL server refers to a server specifically used to perform the ETL (Extract, Transform, Load) process. ETL is a data processing method commonly used in data warehouses, data integration, and data migration. It includes the following three steps: Extraction: extracting data from various data sources (such as databases, file systems, APIs (interfaces), etc.); Transformation: cleaning, transforming, and processing the extracted data to meet the needs of the target system; Loading: loading the transformed data into the target system (such as data warehouses, data lakes, data marts, etc.). In the embodiment of the present application, the above-mentioned ETL server can deploy NAS services.

[0098] S3, the second upload method is that the data to be transmitted is generated by the application system on time according to the business logic, and the sensitive data is stored in the database. The data accuracy can be queried and confirmed according to the conditions on the comprehensive data management system, and then the corresponding nft transmission node number is selected, and the upload application is submitted. The data conditions to be transmitted (time, business type, etc.), node number and file name, etc. are transmitted as parameters to the intelligent management system application through an http request. Then the application reads the data in the database and generates data files according to the parameters, as well as the .lst parameter file required for nft transmission (.lst files usually contain file indexes to indicate which files are transferred) and stores them in the NAS service storage directory.

[0099] S4, after the application successfully stores the file into the NAS service storage, the file upload related information (uploaded file path, file name, etc.), employee ID and log are updated or inserted into the database. The comprehensive data management system can request the business database through http and query the records uploaded by the user on the page to achieve data preview effect.

[0100] S5. After T+2 (T is the upload time of sensitive data), business personnel can query the required sensitive data information through the promotion platform of the employee channel under the pre-established external list (which can be established on the promotion platform of the employee channel).

[0101] Through the embodiments of the present application, the functions, nodes, list numbers and other features for file upload are automatically selectable, and can handle various push requirements and transmission requests from push platforms; and, an automatic file transfer process is implemented between the data sandbox, intelligent management system, and NFT server, opening up a channel for file transmission between the business and head office platforms, and realizing a unified system process for uploading and downloading.

[0102] In addition, the embodiment of the present application has multiple processing methods for file upload, which can process various types of data under different forms and different rules. It can be manually uploaded by the business, or it can be automatically processed according to the business logic and then uploaded after confirmation by the business. The transmission file can be automatically generated according to the rules, and the file can be automatically transmitted and destroyed at the preset time through the script, rather than manual operation of the entire process, avoiding the mistakes and risks that may be caused by manual operation, solving the problem of data uploading to the bank head office platform process as the starting point, and establishing a system to form an automatic transmission file as the end point. Do not perform unnecessary operations, and do not need to manually process the transmission process. Through the comprehensive data management system, the branch intelligent management system, and the nft transmission script, data can be automatically and securely pushed to the head office platform. The above steps are all automated, avoiding mistakes and problems in human operations. The present invention can automatically transmit data files, greatly reducing the investment in human resources, and when the business personnel have high requirements for the timeliness of the data required for promotion or the head office and the demand is relatively frequent, it can be operated at any time, which is conducive to the relevant personnel to quickly obtain data on the platform for business operations.

[0103] It is understandable that in the specific implementation of this application, sensitive data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0104] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0105] According to another aspect of the embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 4 As shown, the device comprises:

[0106] The receiving module 402 is used to receive a data transmission request initiated by the data management system, wherein the data transmission request is used to request the transmission of target data;

[0107] A detection module 404 is used for detecting whether there is a list file in the network shared directory in response to the data transmission request, wherein the list file is used to indicate the data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file;

[0108] The transmission module 406 is used to transmit the data file to the business platform at a preset time according to the list file when the list file is detected in the network shared directory, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

[0109] As an optional solution, the above-mentioned device is also used for: when the above-mentioned list file is detected in the above-mentioned network shared directory, before transmitting the above-mentioned data file to the business platform according to the above-mentioned list file, receiving the above-mentioned data file uploaded in the above-mentioned data management system through a secure file transfer protocol, and storing the above-mentioned data file in the above-mentioned network shared directory; establishing an index of the above-mentioned data file according to the data attributes of the above-mentioned target data, and generating the above-mentioned list file; storing the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0110] As an optional solution, the above-mentioned device is also used for: when the above-mentioned list file is detected in the above-mentioned network shared directory, before transmitting the above-mentioned data file to the business platform according to the above-mentioned list file, in response to the target interaction action occurring in the above-mentioned data management system, determining the target parameters related to the target data, wherein the above-mentioned target parameters are used to indicate the data attributes of the above-mentioned target data, and the above-mentioned target interaction operation is used to indicate obtaining the above-mentioned target data from the business database; initiating a data acquisition request to the business database according to the target parameters in the above-mentioned data transmission request, and obtaining the above-mentioned target data in the above-mentioned business database; generating the above-mentioned data file and the above-mentioned list file according to the above-mentioned target data and the above-mentioned target parameters.

[0111] As an optional solution, the above-mentioned device is used to generate the above-mentioned data file and the above-mentioned list data according to the above-mentioned target data and the above-mentioned target parameters in the following manner: divide the above-mentioned target data according to the business category of the above-mentioned target data to obtain the above-mentioned data file; establish an index of the above-mentioned data file according to the above-mentioned target parameters, generate the above-mentioned list file, and store the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0112] As an optional scheme, the above-mentioned device is also used for at least one of the following: obtaining the list number in the above-mentioned data management system, wherein the above-mentioned list number is used to indicate the upload time and upload type of the above-mentioned data file; receiving the above-mentioned data transmission request initiated by the above-mentioned data management system based on the above-mentioned list number and the transmission node, wherein the above-mentioned transmission node is determined by the business category of the above-mentioned target data; when the above-mentioned target data is in a target state, receiving the above-mentioned data transmission request initiated by the above-mentioned data management system based on the above-mentioned transmission node, wherein the above-mentioned target state indicates that the above-mentioned target data is allowed to be accessed.

[0113] As an optional solution, the above-mentioned device is also used to: periodically run a file generation script to determine whether the data file exists in the network shared directory, wherein the file generation script is deployed on a target server, and the above-mentioned target server allows access to the above-mentioned network shared directory; when the above-mentioned data file exists in the above-mentioned network shared directory, establish an index of the above-mentioned data file according to the above-mentioned target parameters, generate the above-mentioned list file, and store the above-mentioned list file in the above-mentioned network shared directory, wherein the index of the above-mentioned data file is used to indicate the storage path of the above-mentioned data file, and the above-mentioned list file includes the index of the above-mentioned data file and a transmission identifier, and the above-mentioned transmission identifier is used to indicate whether the above-mentioned data file needs to be transmitted to the above-mentioned business platform.

[0114] As an optional solution, the above-mentioned device is also used to: periodically run a file detection script to detect whether the above-mentioned list file exists in the above-mentioned network shared directory, wherein the above-mentioned file detection script is deployed on the target server, the above-mentioned target server allows access to the above-mentioned network shared directory, and the above-mentioned target server is used to periodically transmit the above-mentioned data files to the above-mentioned business platform.

[0115] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0116] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0117] According to one aspect of the present application, a computer program product is provided. The computer program product includes a computer program.

[0118] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0119] Figure 5 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0120] It should be noted that Figure 5 The computer system 500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0121] like Figure 5 As shown, the computer system 500 includes a central processing unit 501 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 502 (ROM) or the program loaded from the storage part 508 to the random access memory 503 (RAM). Various programs and data required for system operation are also stored in the random access memory 503. The central processing unit 501, the read-only memory 502 and the random access memory 503 are connected to each other through a bus 504. The input / output interface 505 (Input / Output interface, i.e., I / O interface) is also connected to the bus 504.

[0122] The following components are connected to the input / output interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a local area network card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read therefrom is installed into the storage section 508 as needed.

[0123] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processor 501, various functions defined in the system of the present application are executed.

[0124] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processor 501, various functions provided by the embodiment of the present application are performed.

[0125] According to another aspect of the embodiment of the present application, an electronic device for implementing the above data processing method is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a terminal device as an example. Figure 6 As shown, the electronic device includes a memory 602 and a processor 604. The memory 602 stores a computer program, and the processor 604 is configured to execute the steps in any of the above method embodiments through the computer program.

[0126] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0127] Optionally, in this embodiment, the above-mentioned processor can be configured to execute the methods in each embodiment of the present application through a computer program.

[0128] Alternatively, a person skilled in the art may understand that: Figure 6 The structure shown is for illustration only. Figure 6 The structure of the electronic device is not limited. Figure 6 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 6 Different configurations are shown.

[0129] Among them, the memory 602 can be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method and device in the embodiments of the present application. The processor 604 executes various functional applications and data processing by running the software programs and modules stored in the memory 602, that is, realizing the above-mentioned data processing method. The memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 602 may further include a memory remotely located relative to the processor 604, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 602 can be specifically, but not limited to, used to store information such as target data. As an example, such as Figure 6 As shown, the memory 602 may include but is not limited to the receiving module 402, the detecting module 404, and the transmitting module 406 in the data processing device. In addition, other module units in the data processing device may also be included but are not limited to, which will not be described in detail in this example.

[0130] Optionally, the transmission device 606 is used to receive or send data via a network. Specific examples of the network may include a wired network and a wireless network. In one example, the transmission device 606 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 606 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0131] In addition, the electronic device further includes: a display 608 for displaying the data file; and a connection bus 610 for connecting various module components in the electronic device.

[0132] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a peer-to-peer network, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0133] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the data processing method provided in various optional implementations of the above-mentioned data processing aspects.

[0134] Optionally, in this embodiment, the above-mentioned computer-readable storage medium can be configured to store data for executing the methods in various embodiments of the present application.

[0135] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0136] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0137] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for one or more electronic devices to execute all or part of the steps of the methods described in each embodiment of the present application.

[0138] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0139] In the several embodiments provided in the present application, it should be understood that the disclosed application can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0142] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data processing method, characterized in that: include: Receiving a data transmission request initiated by a data management system, wherein the data transmission request is used to request transmission of target data; In response to the data transmission request, detecting whether there is a list file in the network shared directory, wherein the list file is used to indicate the data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file; When the list file is detected in the network shared directory, the data file is transmitted to the business platform at a preset time according to the list file, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

2. The method according to claim 1, characterized in that In the case where the list file is detected in the network shared directory, before transmitting the data file to the service platform according to the list file, the method further includes: Receiving the data file uploaded in the data management system through a secure file transfer protocol, and storing the data file in the network shared directory; Establishing the index of the data file according to the data attribute of the target data, and generating the list file; The list file is stored in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

3. The method according to claim 1, characterized in that In the case where the list file is detected in the network shared directory, before transmitting the data file to the service platform according to the list file, the method further includes: In response to a target interaction action occurring in the data management system, a target parameter related to the target data is determined, wherein the target parameter is used to indicate a data attribute of the target data, and the target interaction action is used to indicate obtaining the target data from a business database; Initiate a data acquisition request to a business database according to the target parameters in the data transmission request, and acquire the target data in the business database; The data file and the list file are generated according to the target data and the target parameters.

4. The method according to claim 3, characterized in that The step of generating the data file and the list data according to the target data and the target parameters includes: Divide the target data according to the business category of the target data to obtain the data file; Establish an index of the data file according to the target parameters, generate the list file, and store the list file in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

5. The method according to claim 1, characterized in that The method further comprises at least one of the following: Acquire a list number in the data management system, wherein the list number is used to indicate the upload time and upload type of the data file; receive the data transmission request initiated by the data management system based on the list number and a transmission node, wherein the transmission node is determined by the service category of the target data; In a case where the target data is in a target state, the data transmission request initiated by the data management system based on the transmission node is received, wherein the target state indicates that the target data is allowed to be accessed.

6. The method according to claim 1, characterized in that The method further comprises: Periodically running a file generation script to determine whether the data file exists in the network shared directory, wherein the file generation script is deployed on a target server, and the target server is allowed to access the network shared directory; In the case where the data file exists in the network shared directory, an index of the data file is established according to the target parameters, the list file is generated, and the list file is stored in the network shared directory, wherein the index of the data file is used to indicate the storage path of the data file, and the list file includes the index of the data file and a transmission identifier, and the transmission identifier is used to indicate whether the data file needs to be transmitted to the business platform.

7. The method according to claim 1, characterized in that The method further comprises: A file detection script is periodically run to detect whether the list file exists in the network shared directory, wherein the file detection script is deployed on a target server, the target server is allowed to access the network shared directory, and the target server is used to periodically transmit the data file to the business platform.

8. A data processing device, characterized in that: include: A receiving module, used to receive a data transmission request initiated by a data management system, wherein the data transmission request is used to request transmission of target data; a detection module, configured to detect whether a list file exists in a network shared directory in response to the data transmission request, wherein the list file is used to indicate a data file to be transmitted, the data file includes the target data, the data file is a file uploaded by the data management system, and / or a file obtained by dividing the target data according to the business category of the target data, and the network shared directory is used to store the list file and the data file; A transmission module is used to transmit the data file to the business platform at a preset time according to the list file when the list file is detected in the network shared directory, wherein there is data isolation between the business platform and the data management system, and the business platform is configured to prohibit initiating a request to the data management system to obtain the target data.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein the computer program can be executed by an electronic device to perform the method described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

11. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images