Isolated file submission method and apparatus, computer device, and storage medium

Abstract

The application relates to the technical field of big data, and relates to a cross-library file submission method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining at least one sql processing file for real-time writing information; within a first preset time period, at least one flag identification file corresponding to the sql processing file and having the same file name is obtained; wherein each sql processing file and the corresponding flag identification file belong to the same shared operation folder; according to a preset distributed timing task, the file content of each flag identification file is obtained through traversal; it is judged whether a target mailbox address is contained in a preset text area of the flag identification file; if yes, the sql processing file corresponding to each flag identification file is obtained through traversal; and a task of simultaneously submitting the sql processing file and the corresponding flag identification file is executed. Through the scheme, the problem of low efficiency caused by the fact that non-developers need to communicate with developers who have the right to submit files offline can be solved.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a method, apparatus, computer device, and storage medium for submitting files from different databases. Background Technology

[0002] A cross-database file is a file used for data transfer and conversion between different databases. In software development, standardized continuous integration (CVI) and database deployment pipelines are used to establish a standardized process for team collaboration, code quality assessment, and stability assurance. Standardized CVI refers to a set of standard processes used by developers to develop, submit, compile, and deploy applications using multiple platforms such as VCDS, Gerrit, iCoo, and DevOps. VCDS, as the macro-level development configuration CVI platform, requires code submissions to Gerrit. Jira tasks in iCoo are associated with Gerrit; after code submission, it enters Jenkins for unit testing and DevOps code scanning and evaluation. VCDS's multi-version configuration and scheduled tasks initiate a standardized deployment pipeline to achieve continuous integration. Such conventions and processes ensure that team members continuously contribute code to the service while guaranteeing post-submission stability.

[0003] Currently, developers outside the current team or department do not have the authority to submit files directly. They need to communicate offline to hand the files over to authorized developers for submission, which is inefficient.

[0004] Therefore, there is an urgent need for a method, device, computer equipment, and storage medium for submitting files from different libraries to solve the problem of inefficiency caused by non-developers having to communicate offline with authorized developers to submit files. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer device, and storage medium for submitting files from different libraries, which can solve the problem of inefficiency caused by non-developers needing to communicate offline with authorized developers to submit files.

[0006] Firstly, this application provides a method for submitting files from different libraries. The method includes:

[0007] Obtain at least one SQL processing file for real-time writing of information;

[0008] Within a first preset time period, at least one flag identifier file with the same filename as the SQL processing file is obtained; wherein, each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder;

[0009] According to the preset distributed scheduled task, traverse each flag identification file in the shared operation folder and obtain the file content of each flag identification file;

[0010] Based on the content of each flag file, determine whether the target email address is contained within the preset text area of ​​the flag file;

[0011] If so, then iterate through and retrieve the SQL processing file corresponding to each flag identifier file;

[0012] Based on the contents of each SQL processing file, execute a task that simultaneously submits the SQL processing file and the corresponding flag identification file.

[0013] The submitted SQL processing file and its corresponding flag identification file are transferred from the shared operation folder to the backup folder with the corresponding timestamp.

[0014] In one embodiment, the preset distributed scheduled task includes: at least one distributed file server periodically cross-processing different flag identification files and corresponding SQL processing files in the shared operation folder.

[0015] In one embodiment, if at least one of a first preset condition or a second preset condition is met, the filename of the SQL processing file and the filename of the flag identifier file are modified; the first preset condition includes that the filename of the SQL processing file and the filename of the flag identifier file are not the same as the filename of the shared operation folder, and the second preset condition includes that the filename of the SQL processing file and the filename of the flag identifier file are not the same as the filename of the backup folder.

[0016] In one embodiment, if the SQL processing file corresponding to the flag identifier file is not obtained, the acquisition process is stopped;

[0017] After the second preset time period, the step of traversing and obtaining the SQL processing file corresponding to the flag identifier file is repeated until the acquisition is successful.

[0018] In one embodiment, during the repeated execution of the acquisition step, the cumulative number of times the acquisition step is repeatedly executed is obtained;

[0019] If the cumulative number of executions reaches a preset threshold, the repeated execution process will stop, an email notification of file processing failure will be sent, and the flag file that did not correspond to the obtained SQL processing file will be deleted from the shared operation folder.

[0020] In one embodiment, if not, an email notification of file deletion is sent, and the flag identification file that does not contain the target email address within the preset text area, and the corresponding SQL processing file for the flag identification file, are moved from the shared operation folder to the backup folder with the corresponding timestamp.

[0021] Secondly, this application also provides a cross-library file submission device. The device includes:

[0022] The acquisition module is used to acquire at least one SQL processing file for real-time writing of information;

[0023] The acquisition module is also used to acquire at least one flag identifier file with the same filename as the SQL processing file within a first preset time period; wherein each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder;

[0024] The processing module is used to traverse each flag identifier file in the shared operation folder according to a preset distributed scheduled task and obtain the file content of each flag identifier file.

[0025] The judgment module is used to determine whether the target email address is contained in the preset text area of ​​each flag identifier file based on the file content of each flag identifier file.

[0026] If so, the module is also used to iterate and obtain the SQL processing file corresponding to each flag identifier file;

[0027] The execution module is used to execute tasks that simultaneously submit the SQL processing file and the corresponding flag identification file, based on the contents of each SQL processing file;

[0028] The processing module is also used to transfer the submitted SQL processing file and the corresponding flag identification file from the shared operation folder to the backup folder with the corresponding timestamp.

[0029] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0030] Obtain at least one SQL processing file for real-time writing of information;

[0031] Within a first preset time period, at least one flag identifier file with the same filename as the SQL processing file is obtained; wherein, each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder;

[0032] According to the preset distributed scheduled task, traverse each flag identification file in the shared operation folder and obtain the file content of each flag identification file;

[0033] Based on the content of each flag file, determine whether the target email address is contained within the preset text area of ​​the flag file;

[0034] If so, then iterate through and retrieve the SQL processing file corresponding to each flag identifier file;

[0035] Based on the contents of each SQL processing file, execute a task that simultaneously submits the SQL processing file and the corresponding flag identification file.

[0036] The submitted SQL processing file and its corresponding flag identification file are transferred from the shared operation folder to the backup folder with the corresponding timestamp.

[0037] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0038] Obtain at least one SQL processing file for real-time writing of information;

[0039] Within a first preset time period, at least one flag identifier file with the same filename as the SQL processing file is obtained; wherein, each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder;

[0040] According to the preset distributed scheduled task, traverse each flag identification file in the shared operation folder and obtain the file content of each flag identification file;

[0041] Based on the content of each flag file, determine whether the target email address is contained within the preset text area of ​​the flag file;

[0042] If so, then iterate through and retrieve the SQL processing file corresponding to each flag identifier file;

[0043] Based on the contents of each SQL processing file, execute a task that simultaneously submits the SQL processing file and the corresponding flag identification file.

[0044] The submitted SQL processing file and its corresponding flag identification file are transferred from the shared operation folder to the backup folder with the corresponding timestamp.

[0045] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0046] Obtain at least one SQL processing file for real-time writing of information;

[0047] Within a first preset time period, at least one flag identifier file with the same filename as the SQL processing file is obtained; wherein, each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder;

[0048] According to the preset distributed scheduled task, traverse each flag identification file in the shared operation folder and obtain the file content of each flag identification file;

[0049] Based on the content of each flag file, determine whether the target email address is contained within the preset text area of ​​the flag file;

[0050] If so, then iterate through and retrieve the SQL processing file corresponding to each flag identifier file;

[0051] Based on the contents of each SQL processing file, execute a task that simultaneously submits the SQL processing file and the corresponding flag identification file.

[0052] The submitted SQL processing file and its corresponding flag identification file are transferred from the shared operation folder to the backup folder with the corresponding timestamp.

[0053] The aforementioned method, apparatus, computer equipment, and storage medium for submitting files from different databases avoid the time and cost of offline file transfer by placing pre-release files in an accessible distributed file path. Simultaneously, it enables real-time writing and execution of file content, thereby shortening the deployment process time. By placing release files in a shared operation folder, developers from different teams can submit files and execute tasks within this folder, facilitating inter-team collaboration and communication. By obtaining the flag identifier file corresponding to the SQL processing file and performing judgments and operations based on its content, file consistency is ensured, preventing erroneous submissions and executions. Through pre-defined distributed scheduled tasks, the flag identifier files in the shared operation folder are traversed, and corresponding tasks are executed based on their content, achieving automated task execution. Transferring the submitted SQL processing files and their corresponding flag identifier files to a backup folder with the corresponding timestamp facilitates subsequent traceability and backup, ensuring data reliability and security. In summary, continuous deployment developed collaboratively by different teams can be achieved, improving deployment efficiency, team collaboration, and traceability, and realizing automated task execution, thus achieving the technical objectives. Attached Figure Description

[0054] Figure 1 This is a diagram illustrating the application environment of a cross-library file submission method in one embodiment.

[0055] Figure 2 This is a flowchart illustrating a method for submitting files from different libraries in one embodiment;

[0056] Figure 3 This is a flowchart illustrating the method for submitting files from different libraries in another embodiment;

[0057] Figure 4 This is a structural block diagram of a cross-library file submission device in one embodiment;

[0058] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0060] Currently, developers primarily use the VCDS platform, supplemented by numerous other platforms such as Gerrit, iCoo, and DevOps, for development, submission, compilation, and deployment. This standardized workflow constitutes continuous integration. VCDS serves as the macro-level configuration platform for continuous integration. Application development must proceed through the Gerrit platform to obtain standard code submissions. Jira tasks in the iCoo platform are linked to the Gerrit platform. After code submission, Jenkins unit tests and DevOps perform code scanning and assess the submission score. VCDS's multi-version configuration and scheduled tasks initiate a standardized deployment pipeline, forming continuous integration. Under this agreement, a team can collectively contribute code to the service continuously, ensuring stability after submission.

[0061] Under the rules of "standardized continuous integration," it is generally divided into application continuous integration and data continuous integration. Database deployment is based on multi-version configuration of the VCDS platform, database data source configuration is associated with ledger registration information, and DDL and DML statements for registering and submitting Seas rule constraints are added.

[0062] The heterogeneous file submission method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Server 104 obtains at least one SQL processing file for real-time writing of information. Within a first preset time period, it obtains at least one flag identification file corresponding to the SQL processing file and having the same filename; wherein each SQL processing file and its corresponding flag identification file belong to the same shared operation folder. According to a preset distributed scheduled task, it iterates through each flag identification file in the shared operation folder and obtains the file content of each flag identification file. Based on the file content of each flag identification file, it determines whether a target email address is contained within a preset text area of ​​the flag identification file. If so, it iterates through and obtains the SQL processing file corresponding to each flag identification file. Based on the file content of each SQL processing file, it executes a task that simultaneously submits the SQL processing file and its corresponding flag identification file. Move the submitted SQL processing files and their corresponding flag identification files from the shared operation folder to the backup folder with the corresponding timestamp.

[0063] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0064] In one embodiment, such as Figure 2 As shown, a method for submitting files from different libraries is provided, which can be applied to... Figure 1 Taking the server in the example, the following steps are included:

[0065] Step S202: Obtain at least one SQL processing file for real-time writing of information.

[0066] Specifically, an SQL processing file is a text file (ending in ".sql") used to store and execute database operations, containing SQL statements and commands. By obtaining at least one SQL processing file, information can be written to the database in real time. This SQL processing file can contain a series of SQL statements used to insert, update, or delete data in the database, or to perform other database operations. The specific method of obtaining it may vary depending on the system and environment. SQL processing files for real-time information writing can be obtained in the following ways:

[0067] In a shared operations folder, developers can submit files containing SQL statements, named "sql processing files," which other developers can then retrieve and execute. Developers can share and manage these SQL processing files using version control systems (such as Git), allowing other developers to access the latest versions. In team collaboration tools (such as Jira and Slack), developers can share links or attachments to these SQL processing files, which other developers can then download and execute.

[0068] Step S204: Within a first preset time period, obtain at least one flag identifier file that corresponds to the SQL processing file and has the same filename; wherein each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder.

[0069] Specifically, platform developers upload two files to a designated folder each time, based on different source identifiers. One file is the SQL processing file, and the other is a specially identified flag file (ending with ".flag"). The two files have the same filename but different content. Within a first preset time period, the flag file with the same name as each SQL processing file is located. These SQL processing files and their corresponding flag files all belong to the same shared operations folder. Flag files are typically used to mark the status or progress of an operation or task. In this context, the flag file can be used to indicate the execution status or completion of the SQL processing file.

[0070] First, you need to determine the location and name of the shared operations folder. This folder stores the SQL processing files and flag identification files. Within the shared operations folder, search for the flag identification file with the same filename as each SQL processing file. You can find the corresponding flag identification file by matching filenames. For each SQL processing file, at least one corresponding flag identification file should be found. If multiple flag identification files with the same filename exist, you can select one as the corresponding file.

[0071] For example, the first step is to determine the target path for file uploads, which is the " / usercdata / plmdev / " folder. This path is a shared path on the NAS server with the address 1.2.3.4:9090. The team path is determined based on the different source identifiers. For example, if F-GCMS is the abbreviation for the team, then the files for the F-GCMS team will be uploaded to the " / usercdata / plmdev / F-GCMS / " path. According to the rules, the SQL processing file is uploaded first, followed by the flag identifier file. The filenames are the same, but the content is different. For example, upload the files 20230808120000.sql and 20230808120000.flag. The maximum interval between the uploads of the two files cannot exceed 10 seconds. That is, within 10 seconds, the SQL processing file is uploaded first, followed by the flag file.

[0072] Step S206: According to the preset distributed scheduled task, traverse each flag identifier file in the shared operation folder and obtain the file content of each flag identifier file.

[0073] Specifically, distributed scheduled tasks refer to a management method that integrates scattered, unreliable scheduled tasks into a unified platform, enabling cluster management, scheduling, and distributed deployment. For each flag identifier file, file system APIs or libraries can be used to open the file and read its contents. After obtaining the file content, appropriate processing can be performed. Depending on the requirements, the file content can be stored in a database, used for data analysis, or used to execute specific operations.

[0074] For example, the first step is to determine the folder path, i.e., the path to the shared operations folder where the files are located. This path can be the folder path of any server in the cluster, for example, specified as " / shared_folder / ". Tasks are assigned to six teams (A, B, C, D, E, F), each corresponding to one platform. Each team will process tasks within a specified time period. Distributed scheduled task management ensures that the four servers (a, b, c, d) do not process the same task within the same time period. Distributed scheduling frameworks such as Quartz or Celery can be used to implement the scheduling and allocation of scheduled tasks. Based on the preset distributed scheduled tasks, each flag identifier file in the shared operations folder is traversed within the specified time period for each team. Traversal can be implemented using a programming language or script; a loop structure can be used to traverse each file in the folder. During traversal, for each flag identifier file, the file can be opened and its contents read. The specific reading method depends on the programming language or script used.

[0075] By following the steps above, we can iterate through each flag file in the shared operation folder according to a preset distributed scheduled task and retrieve the content of each flag file. Simultaneously, through distributed scheduled task scheduling, we ensure that multiple servers do not process the same task within the same time period, thus achieving task allocation and scheduling.

[0076] Step S208: Based on the content of each flag identifier file, determine whether the target email address is contained in the preset text area of ​​the flag identifier file.

[0077] Specifically, the flag file may contain other content, but it only needs to determine whether the target email address (e.g., 123456789@mails.icbc) is contained within a preset text region. The preset text region can be defined according to specific needs; it can be a text region between specific lines (e.g., the first line), specific positions, or specific tags in a file. The preset text region is extracted according to a preset method. String manipulation methods can be used, such as extracting text from specific lines, specific positions, or specific tags. For the extracted preset text region, regular expressions or other string matching methods can be used to determine whether it contains the target email address. Matching rules can be written based on the characteristics of the email address, such as specific character combinations and formats. Based on the matching results, appropriate processing can be performed. Matched email addresses can be stored in a database for further operations or to output corresponding prompts.

[0078] Step S210: If yes, then iterate through and obtain the SQL processing file corresponding to each flag identifier file.

[0079] Specifically, if the target email address is found within a preset text area, it is necessary to iterate through and obtain the SQL processing file corresponding to each flag identifier file.

[0080] First, determine the directory or folder containing the SQL processing files. This can be done based on preset rules or paths. Then, use file system APIs or libraries to traverse all files within that directory or folder. A loop can be used to iterate through the files. For each file, determine if it is an SQL processing file, based on its extension or content. If it is, perform the appropriate processing. The obtained SQL processing files can be saved to a list or data structure for later operations. Further operations can be performed on the obtained SQL processing files as needed. For example, statements in the SQL processing files can be executed, the results stored in the database, or other corresponding processing can be performed. Iterate through the list of obtained SQL processing files. For each SQL processing file, the file content can be read and processed according to specific needs. Database operation APIs or libraries can be used to execute SQL statements, store the results in the database, or perform other operations.

[0081] By following the steps above, it is possible to determine whether the target email address is contained within the preset text area of ​​the flag identification file, and if the target email address is contained, to traverse and obtain the SQL processing file corresponding to each flag identification file and perform the corresponding processing.

[0082] Step S212: Based on the contents of each SQL processing file, execute the task of simultaneously submitting the SQL processing file and the corresponding flag identifier file.

[0083] Specifically, based on the obtained SQL processing file content, database operation APIs or libraries can be used to execute SQL statements. Each SQL statement can be treated as a task and executed sequentially or concurrently. The obtained SQL processing file content is traversed. For each SQL statement, the database operation API or library is used to execute the statement. Appropriate processing can be performed based on the execution results, such as error handling and logging.

[0084] Step S214: Transfer the submitted SQL processing file and the corresponding flag identification file from the shared operation folder to the backup folder with the corresponding timestamp, and then send a success email.

[0085] Specifically, as required, the backup folder should be named with a timestamp. A timestamp can be generated based on the current time and used as the name of the backup folder. Date-time related functions or methods can be used to obtain the current timestamp. Using file system APIs or libraries, a folder named with the timestamp can be created under the specified path as the backup folder. File system APIs or libraries can be used to iterate through all files under the shared operation folder.

[0086] You can determine if a file is a submitted SQL processing file and its corresponding flag file based on its filename or other identifiers. For each file that meets the criteria, move it from the shared operations folder to the backup folder. Use file system APIs or libraries to move files that meet the criteria from the shared operations folder to the backup folder. You can use file movement methods to move files from the source path to the target path. You can specify the file location using absolute or relative paths. You can choose whether to keep a copy of the original file or delete the original file, depending on your needs. You can check the results of the file movement to ensure that the file has been successfully moved to the backup folder. You can use file system APIs or libraries to check if a file exists in the backup folder. You can check using absolute or relative paths. You can log or send notifications for subsequent checks and processing.

[0087] Through the above embodiments, placing the release files in an accessible distributed file path avoids the time and cost of offline file transfer, while enabling real-time writing and execution of file content, thereby shortening the deployment process time. By placing the release files in a shared operations folder, developers from different teams can submit files and execute tasks within this folder, achieving collaboration and communication between teams. By obtaining the flag identifier file corresponding to the SQL processing file and judging and operating based on its content, file consistency can be ensured, avoiding erroneous submissions and executions. Through a pre-set distributed scheduled task, the flag identifier files in the shared operations folder are traversed, and corresponding tasks are executed based on their content, achieving automated task execution. Transferring the submitted SQL processing files and their corresponding flag identifier files to a backup folder with the corresponding timestamp facilitates subsequent traceability and backup, ensuring data reliability and security. In summary, continuous deployment developed collaboratively by different teams can be achieved, improving deployment efficiency, team collaboration, and traceability, and realizing automated task execution, thus achieving the technical objectives.

[0088] In one embodiment, the preset distributed scheduled task includes: at least one distributed file server periodically cross-processing different flag identification files and corresponding SQL processing files in a shared operation folder.

[0089] Specifically, a distributed file server is a system, in this embodiment, a NAS server (including multiple clusters), used to store and manage files, providing scalable and highly available file access services. It typically consists of multiple server nodes that can communicate and collaborate via a network, providing a unified file storage and access interface. A shared operations folder is a folder used to store flag identification files and their corresponding SQL processing files. Multiple tasks can access this folder simultaneously to read and process the files within. A scheduled task refers to a specified task that is automatically executed according to a preset time interval or point in time. In this scenario, the scheduled task is used to trigger the file server to process the files in the shared operations folder.

[0090] Cross-processing refers to processing multiple files simultaneously or alternately according to certain rules or strategies. In this scenario, a file server can cross-process different flag files and their corresponding SQL processing files in a shared operation folder. Cross-processing rules can be defined, such as by alphabetical order of filenames or by file creation time. Multithreading, coroutines, message queues, and other technologies can be used to achieve concurrent file processing. Task scheduling and management can be performed based on factors such as task priority and resource availability. The file server can retrieve different flag files and their corresponding SQL processing files from the shared operation folder according to the cross-processing rules and execute the appropriate processing.

[0091] In this embodiment, using a distributed file server improves system availability. When one file server node fails, other nodes can continue to provide file storage and access services, ensuring the normal execution of tasks. Through cross-processing, the system's computing resources can be fully utilized. While a task needs to wait for the processing of a file to complete, other tasks can continue processing other files, avoiding resource waste. Pre-set scheduled tasks allow for flexible adjustment of task execution time and frequency according to actual needs. Different scheduling rules can be set according to business requirements to meet different business scenarios. Because a distributed file server is used, the consistency of files in the shared operation folder is guaranteed across different nodes. When a file is being processed, other nodes will not read or modify the same file simultaneously, avoiding data conflicts and consistency issues.

[0092] In one embodiment, if at least one of a first preset condition or a second preset condition is met, the filename of the SQL processing file and the filename of the flag identifier file are modified; the first preset condition includes that the filename of the SQL processing file and the filename of the flag identifier file are different from the filename of the shared operation folder, and the second preset condition includes that the filename of the SQL processing file and the filename of the flag identifier file are different from the filename of the backup folder.

[0093] Specifically, if the first preset condition is met, namely that the filenames of the SQL processing file and the flag identifier file are different from the filenames in the shared operation folder, the corresponding SQL processing file and flag identifier file can be found based on the filenames in the shared operation folder, and the corresponding processing operations can be performed.

[0094] If the second preset condition is met, namely that the filenames of the SQL processing file and the flag identifier file are different from the filenames in the backup folder, the corresponding SQL processing file and flag identifier file can be found based on the filenames in the backup folder, and the corresponding processing operations can be performed.

[0095] In this embodiment, by making the filenames of the SQL processing file and the flag identifier file different from the filenames of the shared operation folder or the backup folder, naming conflicts can be avoided, ensuring the uniqueness and correctness of the files. This design allows for better file management and processing, avoiding confusion and erroneous operations.

[0096] In one embodiment, reference Figure 3 The method also includes:

[0097] In step S302, if the SQL processing file corresponding to each flag identifier file is not found during the process of traversing and obtaining the SQL processing file, the process is stopped.

[0098] Step S304: After the second preset time period, repeat the step of traversing and obtaining the SQL processing file corresponding to the flag identifier file until successful acquisition.

[0099] Specifically, if the SQL processing file corresponding to the flag identifier file cannot be obtained, it means that the corresponding file was not found in the current attempt. This could be due to the file not existing, an incorrect name, or other reasons. To avoid errors during subsequent operations, this step is designed to immediately stop the acquisition process if the corresponding file is not found. After a second preset time period (e.g., 1 second), the step of traversing and acquiring the SQL processing file corresponding to the flag identifier file is repeated until it is successfully acquired. This means that after the first unsuccessful attempt, a second attempt will be made after the set time period. The purpose of repeated execution is to increase the chance of success and ensure that the required SQL processing file can be obtained. This process will continue to be repeated until the file is successfully acquired.

[0100] In this embodiment, by repeatedly executing the traversal acquisition process, the acquisition operation can be stopped in time during the acquisition process and repeated within a certain time range to ensure that the required SQL processing file can be obtained. This can improve the success rate of file acquisition and avoid subsequent error handling problems caused by failure to obtain the file.

[0101] In one embodiment, during the repeated execution of the acquisition step, the cumulative number of times the acquisition step is repeatedly executed is obtained.

[0102] If the cumulative number of executions reaches the preset threshold, the repeated execution process will stop, an email notification of file processing failure will be sent, and the flag file indicating that no corresponding SQL processing file was obtained will be deleted from the shared operation folder.

[0103] Specifically, if the SQL processing file corresponding to the flag identifier file is not obtained, the acquisition process stops. This is to ensure that subsequent operations are not executed when the corresponding SQL processing file is not found, avoiding error handling. After the second preset time period, the step of traversing and acquiring the SQL processing file corresponding to the flag identifier file is repeated until successful acquisition. This is to continuously attempt to acquire the corresponding SQL processing file within a certain time period until successful acquisition. During the repeated execution of the acquisition step, the cumulative number of executions of the acquisition step is recorded (e.g., 10 times). This is to statistically analyze and monitor the execution status of the acquisition step. If the cumulative number of executions reaches a preset threshold, the repeated execution process stops, an email notification of file processing failure is sent, and the flag identifier file for which the corresponding SQL processing file was not acquired is deleted from the shared operation folder. This is to terminate the acquisition process, notify relevant personnel, and clean up unprocessed identifier files when the acquisition step has repeatedly attempted but still failed to acquire the SQL processing file. Through these steps, under certain circumstances, the acquisition of the SQL processing file corresponding to the flag identifier file can be ensured, and error handling and notification can be performed under certain conditions to ensure the accuracy and reliability of file processing.

[0104] In this embodiment, under certain conditions, the SQL processing file corresponding to the flag identifier file is ensured to be obtained, avoiding the acquisition of incorrect or missed files. By adhering to naming rules for filenames and folder organization, naming conflicts and file confusion can be avoided, improving the accuracy and reliability of file processing. Furthermore, by repeatedly executing the acquisition steps and recording the cumulative execution count, the success rate of file acquisition and the monitoring of the acquisition process can be increased. When the cumulative execution count reaches a preset threshold, a failure notification can be issued promptly, and subsequent processing can be initiated, avoiding prolonged waiting and unnecessary resource waste.

[0105] In one embodiment, after determining whether the preset text area contains the target email address, the method further includes: if not, sending an email notification of file deletion, and moving the flag identification file that does not contain the target email address in the preset text area, and the corresponding SQL processing file of the corresponding flag identification file, from the shared operation folder to the backup folder with the corresponding timestamp.

[0106] Specifically, notify relevant personnel of the file deletion so they can understand and take appropriate measures. Move the flag identification file that does not contain the target email address and its corresponding SQL processing file from the shared operations folder to the backup folder with the corresponding timestamp: This is to remove unprocessed files from the shared operations folder to avoid interfering with subsequent operations, and to back them up to the folder with the corresponding timestamp for later tracking and recovery.

[0107] These operations allow for timely notification of file deletion to relevant personnel and the removal of files not containing the target email address from the shared folder, ensuring the accuracy and reliability of file processing. Simultaneously, these files are backed up to folders with corresponding timestamps, facilitating subsequent tracking and recovery.

[0108] The folder management strategy is as follows:

[0109] Before uploading a file, platform developers check if a file with the same name already exists in the directory; if so, they rename it. A daily scheduled task calculates folder size; if it exceeds 80% of the maximum capacity, developers are notified via email. Capacity will be expanded in 100GB increments in the near future. The default initial maximum capacity is 100GB. When a folder exceeds 90% capacity, it will be locked, prohibiting write operations. Backup flags and SQL files are processed periodically, and files inconsistent with the backup flags are deleted regularly. A distributed file server is introduced; all capability centers or platforms need to connect to the same distributed server and agree on a shared storage server path.

[0110] The following guidelines are established for using shared folders:

[0111] a. Specify the path to the shared operation folder and strictly control read and write permissions. Each platform has its own designated path and user with corresponding permissions. An email warning will be issued for any unauthorized write operations to other platforms.

[0112] b. When the disk reaches its storage limit, the administrator will be notified via email to expand the capacity.

[0113] c. Strictly control uploaded content, do not store redundant or junk data, only allow the storage of DML and DDL statements, and reduce comments.

[0114] d. Uploaded file names can only contain English letters, numbers, underscores, and hyphens, and the content character set is UTF-8, following Unix mode.

[0115] e. Minimize the size of the file content; the content should be concise and useful.

[0116] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0117] Based on the same inventive concept, this application also provides a cross-library file submission device for implementing the cross-library file submission method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more cross-library file submission device embodiments provided below can be found in the limitations of the cross-library file submission method described above, and will not be repeated here.

[0118] In one embodiment, such as Figure 4 As shown, a cross-database file submission device is provided, including: an acquisition module 402, used to acquire at least one SQL processing file for real-time writing information;

[0119] The acquisition module 402 is also used to acquire at least one flag identifier file with the same filename as the SQL processing file within a first preset time period; wherein each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder;

[0120] Processing module 404 is used to traverse each flag identifier file in the shared operation folder according to a preset distributed scheduled task and obtain the file content of each flag identifier file;

[0121] The judgment module 406 is used to determine whether the target email address is contained in the preset text area of ​​each flag identifier file based on the file content of each flag identifier file.

[0122] If so, module 402 is obtained, which is also used to traverse and obtain the SQL processing file corresponding to each flag identifier file;

[0123] Execution module 408 is used to execute a task that simultaneously submits the SQL processing file and the corresponding flag identifier file, based on the contents of each SQL processing file;

[0124] The processing module 404 is also used to transfer the submitted SQL processing file and the corresponding flag identification file from the shared operation folder to the backup folder with the corresponding timestamp.

[0125] In one embodiment, the device further includes: at least one distributed file server periodically cross-processing different flag identification files and corresponding SQL processing files in a shared operation folder.

[0126] In one embodiment, if at least one of the first preset condition or the second preset condition is met, the processing module 404 is further configured to modify the file name of the SQL processing file and the file name of the flag identifier file; the first preset condition includes that the file name of the SQL processing file and the file name of the flag identifier file are different from the file name of the shared operation folder, and the second preset condition includes that the file name of the SQL processing file and the file name of the flag identifier file are different from the file name of the backup folder.

[0127] In one embodiment, the acquisition module 402 is further configured to stop the acquisition process if no SQL processing file corresponding to the flag identifier file is acquired during the process of traversing and acquiring the SQL processing file corresponding to each flag identifier file;

[0128] The execution module 408 is also used to repeatedly execute the step of traversing and obtaining the SQL processing file corresponding to the flag identifier file after the second preset time period until the acquisition is successful.

[0129] In one embodiment, the acquisition module 402 is further configured to acquire the cumulative number of times the acquisition step is repeatedly executed during the repeated execution of the acquisition step;

[0130] The execution module 408 is also used to stop the repeated execution process, send an email notification of file processing failure, and delete the flag identification file that did not correspond to the obtained SQL processing file from the shared operation folder when the cumulative number of executions reaches a preset threshold.

[0131] In one embodiment, if the target email address is not contained in the preset text area, the sending module is further configured to send an email notification of file deletion, and move the flag identification file that does not contain the target email address in the preset text area, and the corresponding SQL processing file of the flag identification file, from the shared operation folder to the backup folder with the corresponding timestamp.

[0132] Each module in the aforementioned heterogeneous file submission device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0133] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for submitting files from different libraries. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0134] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0135] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0136] Obtain at least one SQL processing file for real-time writing of information;

[0137] Within a first preset time period, at least one flag identifier file with the same filename as the SQL processing file is obtained; wherein, each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder;

[0138] According to the preset distributed scheduled task, traverse each flag identification file in the shared operation folder and obtain the file content of each flag identification file;

[0139] Based on the content of each flag file, determine whether the target email address is contained within the preset text area of ​​the flag file;

[0140] If so, then iterate through and retrieve the SQL processing file corresponding to each flag identifier file;

[0141] Based on the contents of each SQL processing file, execute a task that simultaneously submits the SQL processing file and the corresponding flag identification file.

[0142] The submitted SQL processing file and its corresponding flag identification file are transferred from the shared operation folder to the backup folder with the corresponding timestamp.

[0143] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0144] Obtain at least one SQL processing file for real-time writing of information;

[0145] Within a first preset time period, at least one flag identifier file with the same filename as the SQL processing file is obtained; wherein, each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder;

[0146] According to the preset distributed scheduled task, traverse each flag identification file in the shared operation folder and obtain the file content of each flag identification file;

[0147] Based on the content of each flag file, determine whether the target email address is contained within the preset text area of ​​the flag file;

[0148] If so, then iterate through and retrieve the SQL processing file corresponding to each flag identifier file;

[0149] Based on the contents of each SQL processing file, execute a task that simultaneously submits the SQL processing file and the corresponding flag identification file.

[0150] The submitted SQL processing file and its corresponding flag identification file are transferred from the shared operation folder to the backup folder with the corresponding timestamp.

[0151] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0152] Obtain at least one SQL processing file for real-time writing of information;

[0153] Within a first preset time period, at least one flag identifier file with the same filename as the SQL processing file is obtained; wherein, each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder;

[0154] According to the preset distributed scheduled task, traverse each flag identification file in the shared operation folder and obtain the file content of each flag identification file;

[0155] Based on the content of each flag file, determine whether the target email address is contained within the preset text area of ​​the flag file;

[0156] If so, then iterate through and retrieve the SQL processing file corresponding to each flag identifier file;

[0157] Based on the contents of each SQL processing file, execute a task that simultaneously submits the SQL processing file and the corresponding flag identification file.

[0158] The submitted SQL processing file and its corresponding flag identification file are transferred from the shared operation folder to the backup folder with the corresponding timestamp.

[0159] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0160] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0161] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0162] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for submitting files from different libraries, characterized in that, The method includes: Obtain at least one SQL processing file for real-time writing of information; Within a first preset time period, at least one flag identifier file with the same filename as the SQL processing file is obtained; wherein each SQL processing file and the corresponding flag identifier file belong to the same shared operation folder; wherein, in the shared operation folder, the developer submits the SQL processing file, and other developers obtain SQL statements from the SQL processing file and execute them; According to the preset distributed scheduled task, traverse each flag identification file in the shared operation folder and obtain the file content of each flag identification file; Based on the content of each flag file, determine whether the target email address is contained within the preset text area of ​​the flag file; If so, then iterate through and retrieve the SQL processing file corresponding to each flag identifier file; Based on the contents of each SQL processing file, execute a task that simultaneously submits the SQL processing file and the corresponding flag identification file. The submitted SQL processing file and its corresponding flag identification file are transferred from the shared operation folder to the backup folder with the corresponding timestamp.

2. The method according to claim 1, characterized in that, The preset distributed scheduled task includes: at least one distributed file server periodically cross-processing different flag identification files and corresponding SQL processing files in the shared operation folder.

3. The method according to claim 1, characterized in that, The method further includes: If at least one of the first or second preset conditions is met, the filename of the SQL processing file and the filename of the flag identifier file are modified. The first preset condition includes that the filename of the SQL processing file and the filename of the flag identifier file are not the same as the filename of the shared operation folder. The second preset condition includes that the filename of the SQL processing file and the filename of the flag identifier file are not the same as the filename of the backup folder.

4. The method according to claim 1, characterized in that, The process of traversing and obtaining the SQL processing file corresponding to each flag identifier file includes: If the SQL processing file corresponding to the flag identifier file is not obtained, the acquisition process will stop. After the second preset time period, the step of traversing and obtaining the SQL processing file corresponding to the flag identifier file is repeated until the acquisition is successful.

5. The method according to claim 4, characterized in that, The method further includes: During the repeated execution of the acquisition step, the cumulative number of times the acquisition step is repeatedly executed is obtained; If the cumulative number of executions reaches a preset threshold, the repeated execution process will stop, an email notification of file processing failure will be sent, and the flag file that did not correspond to the obtained SQL processing file will be deleted from the shared operation folder.

6. The method according to claim 1, characterized in that, After determining whether the preset text area contains the target email address, the method further includes: If not, send an email notification of file deletion, and move the flag identification file that does not contain the target email address within the preset text area, and the corresponding SQL processing file of the flag identification file, from the shared operation folder to the backup folder with the corresponding timestamp.

7. A cross-database file submission device, characterized in that, The device includes: The acquisition module is used to acquire at least one SQL processing file for real-time writing of information; The acquisition module is also used to acquire at least one flag identification file with the same filename as the SQL processing file within a first preset time period; wherein each SQL processing file and the corresponding flag identification file belong to the same shared operation folder, wherein in the shared operation folder, the developer submits the SQL processing file, and other developers obtain SQL statements from the SQL processing file and execute them; The processing module is used to traverse each flag identifier file in the shared operation folder according to a preset distributed scheduled task and obtain the file content of each flag identifier file. The judgment module is used to determine whether the target email address is contained in the preset text area of ​​each flag identifier file based on the file content of each flag identifier file. If so, the module is also used to iterate and obtain the SQL processing file corresponding to each flag identifier file; The execution module is used to execute tasks that simultaneously submit the SQL processing file and the corresponding flag identification file, based on the contents of each SQL processing file; The processing module is also used to transfer the submitted SQL processing file and the corresponding flag identification file from the shared operation folder to the backup folder with the corresponding timestamp.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images