WEB static file uploading method and device, electronic equipment and readable storage medium
By building directory relationships and using the Diff-utils algorithm for file comparison, the problem of insufficient differences in file upload is solved, efficient and accurate file updates and security is achieved, and the stable operation of the website or system is ensured.
Patent Information
- Application Number
- CN202411222193.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing technology lacks efficient and accurate file differences comparison functions during file upload and release, resulting in website or system abnormalities due to subtle differences in file content.
By building the correspondence between the source directory and the target directory, a List directory tree is generated, and the text type files are deeply compared and analyzed using the Diff-utils algorithm. After generating a comparison report, it is manually confirmed to ensure the accuracy and security of file updates. For non-text-type files, use timestamp suffix to avoid data loss.
It realizes efficient and accurate file differences comparison, ensuring that only files that really need to be updated are uploaded, reducing the amount of data transmitted on the network, improving the upload efficiency, and reducing server resource usage, avoiding misoperation in the automation process.
Smart Images

Figure CN119052228A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communication transmission technology, and in particular to a method, device, electronic device and readable storage medium for uploading a WEB static file. Background Art
[0002] With the rapid development and popularization of Internet technology, the Internet has become the infrastructure of modern society. In order to improve service efficiency, governments, universities, enterprises and other institutions have built portal websites and business systems containing a large number of code files. These code files are written in multiple programming languages and support various functions of websites and systems. As business needs continue to change, these code files need to be updated frequently to ensure the stable operation and functional upgrades of websites and systems. In the current software development and operation and maintenance practices, file uploading and publishing are important links to ensure the update and iteration of website and system functions. However, the traditional file uploading and publishing methods are highly dependent on manual operations, including manual file location, file difference comparison, and execution of upload commands. This method is not only inefficient, but also prone to errors. Faced with a huge file system, manual file difference comparison is not only time-consuming and labor-intensive, but also easy to miss key changes due to negligence, resulting in functional abnormalities of the website or system after release.
[0003] The existing technology mainly uses file upload and publishing tool software to perform automated file replacement, which can reduce the tediousness of manual operations and improve upload efficiency. However, these tools only stay at the simple replacement of the file level and lack in-depth file content difference comparison functions. Even if the file is successfully replaced, if there are slight differences between the replaced file and the expected one (such as code logic errors, inconsistent configuration parameters, etc.), these differences cannot be discovered and corrected in time, which may eventually lead to functional failure or performance degradation of the website or system. In addition, the existing file upload and publishing tools also have deficiencies in version control, rollback mechanism, etc. Once serious problems occur during the publishing process, it is often difficult to quickly locate the source of the problem and effectively roll back to the last stable version, which further increases the complexity and risk of operation and maintenance work. In summary, the existing technology lacks efficient and accurate file difference comparison functions in the file upload and publishing process, and the website may be abnormal due to file differences during the file replacement process.
[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0005] The present application provides a WEB static file uploading method, device, electronic device and readable storage medium to solve the technical problem that the above-mentioned file uploading and publishing methods lack an efficient and accurate file difference comparison function, which leads to website abnormalities due to slight differences in file content during the file replacement process.
[0006] According to one aspect of an embodiment of the present application, the present application provides a WEB static file uploading method, including: S202, constructing a correspondence between a source directory and a target directory and traversing the source directory and the target directory based on the correspondence to generate a corresponding List directory tree; S204, determining whether the current target file exists in the List directory tree, if so, executing S206, otherwise executing S214; S206, judging whether the modification time of the source directory is greater than the modification time of the target directory and determining whether the file size of the corresponding source file is consistent with that of the current target file, if the modification time of the source directory is greater than the modification time of the target directory and the size of the source file is consistent with that of the current target file, executing S204 to traverse the next current target file, otherwise executing S208; S208, judging whether the file type is If it is a text type, execute S210 if it is, otherwise execute S214 after adding a time suffix to the current target file; S210, based on the Diff-utils algorithm, compare and analyze the source directory and the target directory to obtain a first data set and a second data set, and judge based on the second data set if both the source data and the target data are empty, execute S214, otherwise execute S212; S212, query the second data set to obtain data whose source content and target content are not empty and record them in a third data set, output a comparison report based on the third data set combined with a preset comparison report template, and perform manual operation confirmation, if confirmed, execute S214, otherwise execute S206; S214, upload the current target file, and transmit the current target file and the corresponding target directory to the operation end.
[0007] Optionally, the constructing of the correspondence between the source directory and the target directory and traversing the source directory and the target directory based on the correspondence to generate a corresponding List directory tree includes: determining a source directory corresponding to a source file and a target directory corresponding to a target file, the source directory and the target directory respectively including a file type, a file size and a modification time; determining the correspondence between the source directory and the target directory, the correspondence including a sub-directory correspondence and a file correspondence; based on the correspondence, traversing the source directory and the target directory respectively by a recursive algorithm to generate a List directory tree corresponding to the source directory and the target directory.
[0008] Optionally, the correspondence relationship includes a single-node correspondence relationship and a cluster correspondence relationship, the single-node correspondence relationship includes a single source directory corresponding to a target directory, and the cluster correspondence relationship includes a single source directory corresponding to multiple target directories.
[0009] Optionally, after S202, the step further includes caching the List directory tree using a cache technology, specifically including: converting the List directory tree into a serialized format, the serialized format including a string or a byte stream; and storing the serialized List directory tree in a preset cache location.
[0010] Optionally, the determining whether the current target file exists in the List directory tree, if so, executing S206, otherwise executing S214, includes: determining the current target file; traversing all corresponding files in the target directory of the List directory tree, and judging whether the target directory contains the current target file; if the target directory contains the current target file, executing S206; synchronously, if the target directory does not contain the current target file, executing S214 to upload the current target file.
[0011] Optionally, the judging whether the modification time of the source directory is greater than the modification time of the target directory and determining whether the file size of the corresponding source file is consistent with that of the current target file, if the modification time of the source directory is greater than the modification time of the target directory and the size of the source file is consistent with that of the current target file, executing S204 to traverse the next current target file, otherwise executing S208, includes: extracting the modification time and file size of the source directory and the modification time and file size of the corresponding target directory based on the List directory tree; judging whether the modification time of the source directory is greater than the modification time of the target directory to obtain a first comparison result; judging whether the file size of the source directory is equal to the file size of the target directory to obtain a second comparison result; if the first comparison result is yes and the second comparison result is yes, returning to execute S204 to continue traversing the next current target file, otherwise executing S208.
[0012] Optionally, the determination of whether the file type is a text type, if so, executing S210, if not, adding a time suffix to the current target file and then executing S214 includes: segmenting the target file based on a string processing function to obtain the corresponding file extension; determining the type of the target file according to the file extension; if the target file is a text type, executing S210 to use the Diff-utils algorithm to compare and analyze the source file and the target file; if the target file is a non-text type, generating corresponding timestamp information according to the current time, converting the timestamp information into a format and adding it to the target file as a suffix of the target file, and then executing S214.
[0013] Optionally, the source directory and the target directory are compared and analyzed based on the Diff-utils algorithm to obtain a first data set and a second data set, and if the source data and the target data are both empty, S214 is executed based on the second data set, otherwise S212 is executed, including: storing the source directory and the target directory in the cache of the computer; performing data cleaning on the source files corresponding to the source directory and the target files corresponding to the target directory based on the trim method, and the data cleaning at least includes removing blank data; comparing and analyzing the source files in the source directory and the corresponding target files in the target directory line by line based on the Diff-utils algorithm to obtain the same line data and the different line data; constructing a first data set based on each of the same line data, and constructing a second data set based on each of the different line data; judging the source data and the target data in the second data set, and outputting corresponding tag information according to the judgment result; determining whether to construct a comparison report according to the tag information.
[0014] Optionally, the trim method-based data cleaning of the source files corresponding to the source directory and the target files corresponding to the target directory includes: reading the source files in the source directory and the target files in the target directory; extracting each line of character strings corresponding to the source files and the target files; removing blank characters in the character strings by the trim method; storing the character strings after removing the blank characters in a temporary file until the blank characters in each line of character strings in the source files and the target files are removed to obtain the source files and the target files after data cleaning.
[0015] Optionally, the marking information includes a first mark, a second mark and a third mark, and outputting the corresponding marking information according to the judgment result includes: outputting the first mark if the source data is empty, outputting the second mark if the target data is empty, and outputting the third mark if both the source data and the target data are not empty; if the marking information includes the first mark and / or the second mark, executing S214 to directly upload the current target file; synchronously, if the marking information includes the third mark, executing S212 to build a comparison report.
[0016] Optionally, the query on the second data set obtains data in which both the source content and the target content are not empty and records them in a third data set, outputs a comparison report based on the third data set in combination with a preset comparison report template, and performs manual confirmation. If confirmed, execute S214, otherwise execute S206, including: filtering the second data set to obtain data in which both the source data and the target data are not empty; storing the data in which both the source data and the target data obtained from the query are not empty in the third data set; obtaining a preset comparison report template, filling the data in the third data set into the comparison report template to obtain a comparison report; performing manual confirmation on the comparison report, if the confirmation is successful, output the comparison report and then execute S214, otherwise execute S206.
[0017] Optionally, uploading the current target file and transmitting the current target file and the corresponding target directory to the operation end includes: uploading the current target file and determining the directory corresponding to the current target file; recording the current target file and the directory corresponding to the current target file; and publishing the recorded current target file and the corresponding directory to the corresponding operation end in a preset format.
[0018] According to another aspect of the embodiment of the present application, the present application provides a WEB static file uploading device, including: a directory construction module, used to construct a correspondence between a source directory and a target directory and traverse the source directory and the target directory based on the correspondence to generate a corresponding List directory tree; a target file checking module, used to determine whether the current target file exists in the List directory tree, if so, execute S206, otherwise execute S214; a file status judgment module, used to determine whether the modification time of the source directory is greater than the modification time of the target directory and determine whether the file size of the source file is consistent with that of the current target file, if the modification time of the source directory is greater than the modification time of the target directory and the size of the source file is consistent with that of the current target file, execute S204 to traverse the next current target file, otherwise execute S208; a file type judgment module, used to determine whether the file type is If it is not a text type, then execute S210; if not, add a time suffix to the current target file and then execute S214; a file comparison module, used to compare and analyze the source directory and the target directory based on the Diff-utils algorithm to obtain a first data set and a second data set, and judge based on the second data set if both the source data and the target data are empty, then execute S214; if not, execute S212; a difference data processing module, used to query the second data set to obtain data whose source content and target content are not empty and record them in a third data set, output a comparison report based on the third data set combined with a preset comparison report template, and perform manual operation confirmation, if confirmed, execute S214, otherwise execute S206; a file upload module, used to upload the current target file, and transmit the target file and the corresponding target directory to the operation end.
[0019] According to another aspect of an embodiment of the present application, the present application provides an electronic device, including a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the steps of the above method are implemented when the processor executes the computer program.
[0020] According to another aspect of an embodiment of the present application, the present application also provides a computer-readable medium having a non-volatile program code executable by a processor, and the program code enables the processor to execute the above method.
[0021] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art:
[0022] This application can automatically detect files that need to be updated by constructing a correspondence between the source directory and the target directory, and traversing and generating a List directory tree. It not only determines whether an update is needed based on whether the file exists or the file size, but also further considers the modification time of the file, avoiding unnecessary file transfers, improving upload efficiency, and ensuring that only files that really need to be updated are processed. For text-type files, the Diff-utils algorithm is used for detailed difference comparison and analysis, which helps to quickly determine the specific content of the file changes, so as to determine whether it needs to be uploaded. For non-text files, a simple timestamp suffix method is adopted to avoid data loss problems that may be caused by direct overwriting. After detecting file differences, the file is not directly replaced, but a comparison report is generated first, and manual confirmation is performed to ensure the accuracy and security of file updates and avoid possible misoperations during the automation process. This solution builds a correspondence between the source directory and the target directory, and uses the Diff-utils algorithm to perform in-depth comparative analysis of text type files. It can efficiently and accurately identify the differences in file content. This difference comparison method ensures the accuracy of file updates, improves the efficiency and accuracy of file difference comparison, and ensures that only files that really need to be updated are uploaded, reducing the amount of data transmitted over the network, improving upload efficiency, and also reducing the occupancy of server resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0025] Figure 1 A schematic diagram of the hardware environment of a WEB static file uploading method provided according to an embodiment of the present application;
[0026] Figure 2 A schematic diagram of a WEB static file uploading method according to an embodiment of the present application;
[0027] Figure 3 A block diagram of a WEB static file uploading device provided according to an embodiment of the present application;
[0028] Figure 4 A schematic diagram of an optional electronic device structure provided in an embodiment of the present application. DETAILED DESCRIPTION
[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0030] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.
[0031] In the related art, the file uploading and publishing methods lack an efficient and accurate file difference comparison function, and during the file replacement process, slight differences in file content may cause website abnormalities.
[0032] In order to solve the problems mentioned in the background technology, according to one aspect of the embodiments of the present application, an embodiment of an autonomous learning spam filtering method is provided.
[0033] Optionally, in the embodiment of the present application, the above method can be applied to Figure 1 In the hardware environment composed of the terminal 101 and the server 103 shown in FIG. Figure 1 As shown, the server 103 is connected to the terminal 101 via a network, and can be used to provide services for the terminal or a client installed on the terminal. A database 105 can be set on the server or independently of the server to provide data storage services for the server 103. The above-mentioned network includes but is not limited to: a wide area network, a metropolitan area network or a local area network, and the terminal 101 includes but is not limited to a PC, a mobile phone, a tablet computer, etc.
[0034] A method for uploading a static WEB file in the embodiment of the present application can be executed by the server 103, or can be executed by the server 103 and the terminal 101 together, such as Figure 2 As shown, the method may include the following steps:
[0035] S202: Construct a correspondence between the source directory and the target directory and traverse the source directory and the target directory based on the correspondence to generate a corresponding List directory tree.
[0036] Specifically, S202 includes:
[0037] Determine a source directory corresponding to a source file and a target directory corresponding to a target file, wherein the source directory and the target directory respectively include file type, file size and modification time;
[0038] Determine the correspondence between the source directory and the target directory, including the correspondence between subdirectories and files;
[0039] Based on the corresponding relationship, the source directory and the target directory are traversed respectively through a recursive algorithm to generate a List directory tree corresponding to the source directory and the target directory.
[0040] Furthermore, the correspondence relationship includes a single-node correspondence relationship and a cluster correspondence relationship. The single-node correspondence relationship includes a single source directory corresponding to a target directory, and the cluster correspondence relationship includes a single source directory corresponding to multiple target directories.
[0041] Furthermore, after S202, the process also includes caching the List directory tree using a cache technology (caching technology), which specifically includes:
[0042] Convert the List directory tree into a serialized format, which includes a string or a byte stream;
[0043] Store the serialized List directory tree to the preset cache location.
[0044] It can be understood that this embodiment is mainly used for uploading WEB static files. WEB files are files stored on a Web server and can be accessed and downloaded by users through the HTTP protocol or other Web protocols. In this embodiment, first obtain the corresponding source file and target file, determine the source directory corresponding to the source file, and determine the target directory corresponding to the target file. The directory includes file type, file size and modification time, wherein the file type includes, for example, html, css and zip, and the modification time is set to the modification date. Traverse each line in the source directory and the target directory respectively to determine the corresponding relationship between the source directory and the target directory. It can be a one-to-one node correspondence or a one-to-many cluster correspondence, for example, "D: / website" corresponds to "website" on the "192.168.*.*" server, and the cluster correspondence is mainly for cluster-deployed Web applications. Then, based on the corresponding relationship, the source directory and the target directory are sorted into a table corresponding to each other in each row, that is, a List directory tree, see the directory tree relationship table shown in Table 1. The source directory includes subdirectory 1, subdirectory 2, subdirectory 3, and subdirectory 4. Subdirectory 1 of the source directory includes files 1.1 and 1.2. Subdirectory 2 of the source directory includes file 2.1. Subdirectory 3 of the source directory includes files 3.1 and 3.2. There is no file in subdirectory 4 of the source directory. The target directory includes subdirectory 1, subdirectory 2, and subdirectory 3. Subdirectory 1 of the target directory includes files 1.1 and 1.2. Subdirectory 2 of the target directory includes file 2.1. Subdirectory 3 of the target directory includes file 3.1. After the List directory tree is constructed, the obtained List directory tree is cached.
[0045] Table 1 List directory tree relationship table
[0046] Source directory (file type, size, modification time) Target directory (file type, size, modification time) ...Subdirectory 1 ...Subdirectory 1 ......File 1.1 ......File 1.1 ...File 1.2 ...File 1.2 ...Subdirectory 2 ...Subdirectory 2 ......File 2.1 ......File 2.1 ...Subdirectory 3 ...Subdirectory 3 ......File 3.1 ......File 3.1 ...File 3.2 ......none ...Subdirectory 4 ...none ...... ......none
[0047] In the above embodiment, by constructing the correspondence between the source directory and the target directory, and traversing and generating the corresponding List directory tree based on the correspondence, it is possible to clearly determine which files or directories need to be copied, updated or ignored, greatly improving the efficiency of data synchronization. During the synchronization process, taking into account attributes such as file type, file size and modification time, it is possible to ensure that the data between the source directory and the target directory are highly consistent, which helps to avoid data loss or inconsistency. By introducing single-node correspondence and cluster correspondence, it is possible to flexibly respond to various complex synchronization requirements. Using cache technology to cache the List directory tree can significantly reduce repeated calculations and resource consumption, so that in the subsequent synchronization process, the directory tree information can be read directly from the cache without re-traversing the entire directory structure, thereby greatly improving the system response speed and throughput. The List directory tree is converted into a serialized format and stored in a preset cache location. This cache mechanism has good scalability and maintainability, and can easily adjust the cache strategy or increase the cache capacity to meet new needs.
[0048] S204, determine whether the current target file exists in the List directory tree, if so, execute S206, otherwise execute S214.
[0049] Specifically, S204 includes:
[0050] Determine the current target file;
[0051] Traverse all the corresponding files in the target directory of the List directory tree to determine whether the target directory contains the current target file;
[0052] If the target directory contains the current target file, execute S206;
[0053] Synchronously, if the target directory does not contain the current target file, S214 is executed to upload the current target file.
[0054] In the present embodiment, first determine the current target file according to actual needs, call the List directory tree that is pre-built and cached, start from the root node of the List directory tree, traverse the target directory branch level by level, until reaching the leaf node (i.e., a specific file or directory). During the traversal process, for each file node reached, it is compared with the name of the current target file. If a matching file name is found, and the file size, modification time and other attributes of the file also meet the preset conditions, it is determined that the current target file is included in the target directory found. When setting "Whether to include the current target file" to empty, output "1", setting "Whether to include the current target file" to not empty is output "0", when output "1", execute to transfer to step S214 to upload the file, when output "0", execute subsequent steps.
[0055] In the above embodiment, by pre-building and caching the List directory tree, files in the target directory can be quickly retrieved without having to re-traverse the entire directory structure each time synchronization occurs. This caching mechanism significantly improves the efficiency of file existence checks, thereby speeding up the synchronization process. When determining whether a file exists, not only the file name match is considered, but also the file size, modification time and other attributes are considered, ensuring the accuracy of synchronization and reducing data inconsistency problems caused by misjudgment. During the traversal process, if the current target directory or the corresponding parent directory is empty (i.e., the node corresponding to the branch in the List directory tree is empty), the upload operation is directly performed, so that the system can flexibly respond to various boundary conditions.
[0056] S206. Determine whether the modification time of the source directory is greater than the modification time of the target directory and determine whether the file size of the corresponding source file is consistent with that of the current target file. If the modification time of the source directory is greater than the modification time of the target directory and the size of the source file is consistent with that of the current target file, execute S204 to traverse the next current target file, otherwise execute S208.
[0057] Specifically, S206 includes:
[0058] Extract the modification time and file size of the source directory and the modification time and file size of the corresponding target directory based on the List directory tree;
[0059] Determine whether the modification time of the source directory is greater than the modification time of the target directory to obtain a first comparison result;
[0060] Determine whether the file size of the source directory is equal to the file size of the target directory to obtain a second comparison result;
[0061] If the first comparison result is yes and the second comparison result is yes, the process returns to S204 to continue traversing the next current target file; otherwise, the process executes S208.
[0062] In this embodiment, firstly, the modification time and file size of the current target file in the source directory and the target directory are extracted from the List directory tree, and it is determined whether the modification time of the file in the source directory is greater than the modification time of the corresponding file in the target directory. If so, the first comparison result is recorded as "yes", otherwise it is "no". Then it is determined whether the file size of the file in the source directory is equal to the file size of the corresponding file in the target directory. If so, the second comparison result is recorded as "yes", otherwise it is "no". If the first comparison result is "yes" and the second comparison result is "yes", then "1" is output. At this time, the file in the target directory is already the latest, and no operation is required. It directly returns to execute S204 to continue traversing the next current target file. If any comparison result is "no", then "0" is output, and S208 is executed for further processing.
[0063] In the above embodiment, by comparing the modification time and file size of the files in the source directory and the target directory, it is possible to intelligently determine which files need to be synchronized. If the files in the target directory are already the latest (i.e., the modification time is later than or equal to the files in the source directory, and the file size is consistent), there is no need to transfer data, thereby saving network bandwidth and storage resources. Since it is possible to quickly identify which files need to be synchronized, these files can be processed first, thereby improving the overall efficiency of the synchronization process. In addition, by reducing unnecessary file checks and transmission operations, synchronization tasks can be completed more quickly, shortening the user's waiting time. By comparing the modification time and size of the files, it is possible to more accurately determine which files have been modified, and accordingly decide whether the files in the target directory need to be updated, which helps reduce the risk of data conflicts and ensures the accuracy and reliability of the synchronization process.
[0064] S208, determine whether the file type is a text type, if so, execute S210, if not, add a time suffix to the current target file and then execute S214.
[0065] Specifically, S208 includes:
[0066] Split the target file based on the string processing function to obtain the corresponding file extension;
[0067] Determine the type of the target file based on the file extension;
[0068] If the target file is of text type, S210 is executed to compare and analyze the source file and the target file using the Diff-utils algorithm (difference analysis algorithm);
[0069] If the target file is of a non-text type, corresponding timestamp information is generated according to the current time, and the timestamp information is format-converted and added to the target file as a suffix of the target file, and then S214 is executed.
[0070] In this embodiment, text type files include various types of files such as js, css, jsp, php, etc. If the type of the target file is determined to be a text type, "1" is output, and S210 is continued to be executed to compare the source file with the target file. If the type of the target file is determined to be a non-text type, "0" is output, indicating that it is a non-unmatchable file, and a time suffix is added to the target file, such as "yyyyMMddHHmmss", and then S214 is executed to upload the file.
[0071] S210, based on the Diff-utils algorithm, compare and analyze the source directory and the target directory to obtain a first data set and a second data set, and determine based on the second data set that if both the source data and the target data are empty, execute S214, otherwise execute S212.
[0072] Specifically, S210 includes:
[0073] Store the source and target directories in the computer's cache;
[0074] Based on the trim method, data cleaning is performed on the source files corresponding to the source directory and the target files corresponding to the target directory, where the data cleaning at least includes removing blank data;
[0075] Based on the Diff-utils algorithm, the source files in the source directory are compared and analyzed line by line with the corresponding target files in the target directory to obtain the same line data and the different line data;
[0076] Constructing a first data set based on identical rows of data, and constructing a second data set based on different rows of data;
[0077] Judging the source data and the target data in the second data set, and outputting corresponding marking information according to the judging result;
[0078] Determine whether to construct a comparison report based on the tag information.
[0079] In this embodiment, the format of "target file + source file" is first output to store the target file and the source file in the computer, and then the target file and the source file stored in the computer are cleaned, and then the [source file line] and the [target file line] are traversed line by line. For example, when traversing to the nth line, if "[source file line n] == [target file line n]", the same data is stored in the first data set, if "[source file line n] != [target file line n]", the different data are stored in the second data set, and finally the second data set is used to output the marking information of the comparison result.
[0080] Furthermore, data cleaning is performed on the source files corresponding to the source directory and the target files corresponding to the target directory based on the trim method, including:
[0081] Read the source files in the source directory and the target files in the target directory;
[0082] Extract each line of string corresponding to the source file and the target file;
[0083] Use the trim method to remove blank characters from the string;
[0084] The string after removing blank characters is stored in a temporary file until the blank characters of each line of the string in the source file and the target file are removed to obtain the source file and the target file after data cleaning.
[0085] In this embodiment, firstly, the source file in the source directory and the target file in the target directory are read respectively, each line of the source file and the target file is traversed, each line in the file is treated as an independent string for processing, the string content is extracted from the current line, and the trim method is used to remove the blank characters at both ends of the current string, and the blank characters include space, tab, line break, etc. The string after removing the blank characters is stored in a temporary file, and the above steps are repeated until each line of the string in the source file and the target file is processed, and finally all the processed strings (i.e., the lines after removing the blank characters) are reassembled into files in the original order, and the original source file and the target file are overwritten or replaced, so as to obtain the data cleaned version.
[0086] In the above embodiment, by removing the blank characters at both ends of the string, unnecessary spaces, tabs or line breaks that may be introduced by different editors or systems when saving files can be eliminated, which helps to ensure the consistency of the source file and the target file at the content level. Blank characters take up extra space during data storage and transmission. Removing redundant blank characters can save storage space and may improve the efficiency of data processing. In subsequent data processing, analysis or synchronization, removing blank characters can reduce errors caused by blank characters. For example, when comparing strings, additional blank characters may cause originally identical strings to be considered different, thereby affecting the accuracy of the results.
[0087] Specifically, the marking information includes a first marking, a second marking, and a third marking, and the marking information corresponding to the output according to the judgment result includes:
[0088] If the source data is empty, the first mark is output; if the target data is empty, the second mark is output; if both the source data and the target data are not empty, the third mark is output;
[0089] If the marking information includes the first marking and / or the second marking, executing S214 to directly upload the current target file;
[0090] Simultaneously, if the marking information includes the third marking, S212 is executed to construct a comparison report.
[0091] In this embodiment, the first tag is set to "delete", the second tag is set to "insert", and the third tag is set to "change", and the Diff-utils algorithm is used to compare and analyze the source directory and the target directory. Check the generated tag information. If it contains at least one of "delete" and "insert", execute S214 to upload the current target file. If it contains "change", execute S212 to build a comparison report.
[0092] In the above embodiment, by setting a variety of different tags such as "delete", "insert" and "change" to indicate the specific differences between the source data and the target data, subsequent data processing can be performed based on more accurate information, thereby improving the flexibility and accuracy of data processing.
[0093] S212, query the second data set to obtain data whose source content and target content are not empty and record them in the third data set, output a comparison report based on the third data set combined with a preset comparison report template, and perform manual operation confirmation, if confirmed, execute S214, otherwise execute S206.
[0094] Specifically, S212 includes:
[0095] Performing a filtering query on the second data set to obtain data where both the source data and the target data are not empty;
[0096] The data obtained by querying and whose source data and target data are not empty are stored in a third data set;
[0097] Obtaining a preset comparison report template, and filling the data in the third data set into the comparison report template to obtain a comparison report;
[0098] Perform manual confirmation operation on the comparison report. If the confirmation is successful, the comparison report is output and then S214 is executed. Otherwise, S206 is executed.
[0099] In this embodiment, the second data set is first filtered and queried to query data records whose source data and target data are not empty. The second data set is traversed through programming logic to check whether the source data field and target data field of each record are not empty. If so, the found data record is added to the new third data set, and the third data set will only contain data records with content differences. After the third data set is constructed, the preset comparison report template is loaded, and then the third data set is traversed, and each data record in the third data set is inserted into the corresponding project column of the comparison report template to be saved in a specific format to obtain the final comparison report, which can be saved in Word, PDF, HTML and other document formats according to requirements. After obtaining the comparison report, it is necessary to perform a manual confirmation operation to verify whether the file in the comparison report can be uploaded, and push the comparison report to the corresponding manual review terminal, which allows the user to view the comparison report and enter confirmation information. If the user confirms that the content of the comparison report is correct and the file in the comparison report can be uploaded, S214 is executed to upload the file. If the user does not confirm that the file in the report cannot be uploaded, S206 is executed to re-traverse and analyze the next current target file.
[0100] In the above embodiment, by filtering and querying the second data set, data whose source content and target content are not empty are specifically screened out, ensuring that the content of the comparison report is generated based on real and effective data changes, thereby enhancing the accuracy and reliability of data processing. The comparison report is generated based on the screened data (i.e., the third data set) combined with the preset comparison report template, which can clearly and accurately display the detailed information of the data changes, thereby improving the practicality and readability of the comparison report. By introducing the manual operation confirmation link, the controllability of data changes is effectively guaranteed, and the data risks caused by misoperation or misjudgment are reduced.
[0101] S214: Upload the current target file, and transfer the current target file and the corresponding target directory to the operation terminal.
[0102] Specifically, S214 includes:
[0103] Upload the current target file and determine the directory corresponding to the current target file;
[0104] Record the current target file and the directory corresponding to the current target file;
[0105] Publish the recorded current target file and the corresponding directory to the corresponding operation terminal according to the preset format.
[0106] In this embodiment, the current target file to be uploaded is first found, and sent to the transmission queue for uploading, and a corresponding data structure or temporary file is constructed to record the current target file and its corresponding directory information. The data structure can be set as a hash table or a list. Finally, the recorded current target file and the corresponding directory information are packaged according to a preset format to obtain a corresponding file package, for example, they can be packaged in a file format such as JSON or XML, and then sent to the operation end through the corresponding network protocol.
[0107] In the above embodiment, by uploading the target file and its corresponding directory, it is possible to ensure that the file system structure reconstructed on the operating end is consistent with the source system, thereby avoiding the problem of data inconsistency. Recording and packaging the file and its directory information during the transmission process can reduce the number of transmissions and the amount of data transmitted. Especially when processing a large number of small files, this batch processing method can significantly improve the synchronization efficiency.
[0108] According to another aspect of the embodiment of the present application, Figure 3 As shown, a WEB static file uploading device is provided, comprising:
[0109] The directory construction module 301 is used to construct a corresponding relationship between the source directory and the target directory and traverse the source directory and the target directory based on the corresponding relationship to generate a corresponding List directory tree;
[0110] The target file checking module 303 is used to determine whether the current target file exists in the List directory tree, and if so, execute S206, otherwise execute S214;
[0111] The file status judgment module 305 is used to judge whether the modification time of the source directory is greater than the modification time of the target directory and determine whether the file size of the source file is consistent with that of the current target file. If the modification time of the source directory is greater than the modification time of the target directory and the size of the source file is consistent with that of the current target file, S204 is executed to traverse the next current target file, otherwise S208 is executed.
[0112] The file type determination module 307 is used to determine whether the file type is a text type, if so, execute S210, if not, add a time suffix to the current target file and then execute S214;
[0113] The file comparison module 309 is used to compare and analyze the source directory and the target directory based on the Diff-utils algorithm to obtain a first data set and a second data set, and to judge based on the second data set that if both the source data and the target data are empty, then S214 is executed, otherwise, S212 is executed;
[0114] The difference data processing module 311 is used to query the second data set to obtain data whose source content and target content are not empty and record them in the third data set, output a comparison report based on the third data set combined with a preset comparison report template, and perform manual operation confirmation. If confirmed, execute S214, otherwise execute S206;
[0115] The file upload module 313 is used to upload the current target file and transmit the target file and the corresponding target directory to the operation end.
[0116] It should be noted that the directory construction module 301 in this embodiment can be used to execute step S202 in the embodiment of the present application, the target file checking module 303 in this embodiment can be used to execute step S204 in the embodiment of the present application, the file status judgment module 305 in this embodiment can be used to execute step S206 in the embodiment of the present application, the file type judgment module 307 in this embodiment can be used to execute step S208 in the embodiment of the present application, the file comparison module 309 in this embodiment can be used to execute step S210 in the embodiment of the present application, the difference data processing module 311 in this embodiment can be used to execute step S212 in the embodiment of the present application, and the file upload module 313 in this embodiment can be used to execute step S214 in the embodiment of the present application.
[0117] According to another aspect of the embodiments of the present application, the present application provides an electronic device, such as Figure 4 As shown, it includes a memory 402, a processor 403, a communication interface 404 and a communication bus 405. The memory 402 stores a computer program that can be run on the processor 404. The memory 402 and the processor 404 communicate through the communication interface 406 and the communication bus 408. When the processor 404 executes the computer program, the steps of the above method are implemented.
[0118] The electronic device provided in the embodiment of the present application may specifically be a module capable of realizing a communication function or a terminal device including the module, etc. The terminal device may be a mobile terminal or a smart terminal. The mobile terminal may specifically be at least one of a mobile phone, a tablet computer, a laptop computer, etc.; the smart terminal may specifically be a terminal including a wireless communication module such as a smart car, a smart watch, a shared bicycle, a smart cabinet, etc.; the module may specifically be a wireless communication module, such as any one of a 2G communication module, a 3G communication module, a 4G communication module, a 5G communication module, and a NB-IOT communication module.
[0119] The memory and processor in the above electronic device communicate through the communication bus and the communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, etc.
[0120] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0121] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0122] According to another aspect of the embodiments of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any of the above embodiments.
[0123] Optionally, in an embodiment of the present application, the computer-readable medium is configured to store program codes for the processor to execute the following steps:
[0124] S202, constructing a correspondence relationship between the source directory and the target directory and traversing the source directory and the target directory based on the correspondence relationship to generate a corresponding List directory tree;
[0125] S204, determine whether the current target file exists in the List directory tree, if so, execute S206, otherwise execute S214;
[0126] S206, judging whether the modification time of the source directory is greater than the modification time of the target directory and determining whether the file size of the corresponding source file is consistent with that of the current target file, if the modification time of the source directory is greater than the modification time of the target directory and the size of the source file is consistent with that of the current target file, executing S204 to traverse the next current target file, otherwise executing S208;
[0127] S208, determine whether the file type is a text type, if yes, execute S210, if no, add a time suffix to the current target file and execute S214;
[0128] S210, performing comparative analysis on the source directory and the target directory based on the Diff-utils algorithm to obtain a first data set and a second data set, and performing a judgment based on the second data set if both the source data and the target data are empty, then executing S214, otherwise executing S212;
[0129] S212, query the second data set to obtain data whose source content and target content are not empty and record them in the third data set, output a comparison report based on the third data set in combination with a preset comparison report template, and perform manual operation confirmation, if confirmed, execute S214, otherwise execute S206;
[0130] S214: Upload the current target file, and transfer the current target file and the corresponding target directory to the operation terminal.
[0131] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.
[0132] When the embodiments of the present application are specifically implemented, reference may be made to the above-mentioned embodiments, which have corresponding technical effects.
[0133] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions of the present application, or a combination thereof.
[0134] For software implementation, the technology of this article can be implemented by a unit that performs the functions of this article. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.
[0135] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0136] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0137] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0138] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0139] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0140] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a ..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.
[0141] The above is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A method for uploading a static file on a web, characterized in that: include: S202, constructing a correspondence relationship between a source directory and a target directory and traversing the source directory and the target directory based on the correspondence relationship to generate a corresponding List directory tree; S204, determining whether the current target file exists in the List directory tree, if so, executing S206, otherwise executing S214; S206, judging whether the modification time of the source directory is greater than the modification time of the target directory and determining whether the file size of the corresponding source file is consistent with that of the current target file, if the modification time of the source directory is greater than the modification time of the target directory and the size of the source file is consistent with that of the current target file, executing S204 to traverse the next current target file, otherwise executing S208; S208, determine whether the file type is a text type, if yes, execute S210, if no, add a time suffix to the current target file and execute S214; S210, performing comparative analysis on the source directory and the target directory based on the Diff-utils algorithm to obtain a first data set and a second data set, and performing a judgment based on the second data set if both the source data and the target data are empty, then executing S214, otherwise executing S212; S212, query the second data set to obtain data whose source content and target content are not empty and record them in the third data set, output a comparison report based on the third data set combined with a preset comparison report template, and perform manual operation confirmation, if confirmed, execute S214, otherwise execute S206; S214: Upload the current target file, and transmit the current target file and the corresponding target directory to the operation end.
2. A method for uploading a WEB static file according to claim 1, characterized in that: The step of constructing a correspondence between a source directory and a target directory and traversing the source directory and the target directory based on the correspondence to generate a corresponding List directory tree includes: Determine a source directory corresponding to the source file and a target directory corresponding to the target file, wherein the source directory and the target directory respectively include the file type, file size and modification time; Determine a correspondence between the source directory and the target directory, wherein the correspondence includes a subdirectory correspondence and a file correspondence; Based on the corresponding relationship, the source directory and the target directory are traversed respectively by a recursive algorithm to generate a List directory tree corresponding to the source directory and the target directory.
3. A WEB static file uploading method according to claim 1, characterized in that: The step of determining whether the current target file exists in the List directory tree, and if so, executing S206, and otherwise executing S214, includes: Determine the current target file; Traverse all the files corresponding to the target directory in the List directory tree to determine whether the target directory contains the current target file; If the target directory contains the current target file, execute S206; Synchronously, if the target directory does not contain the current target file, S214 is executed to upload the current target file.
4. A method for uploading a WEB static file according to claim 1, characterized in that: The step of judging whether the modification time of the source directory is greater than the modification time of the target directory and determining whether the file size of the corresponding source file is consistent with that of the current target file, if the modification time of the source directory is greater than the modification time of the target directory and the size of the source file is consistent with that of the current target file, executing S204 to traverse the next current target file, otherwise executing S208, including: Extracting the modification time and file size of the source directory and the modification time and file size of the corresponding target directory based on the List directory tree; Determine whether the modification time of the source directory is greater than the modification time of the target directory to obtain a first comparison result; Determining whether the file size of the source directory is equal to the file size of the target directory to obtain a second comparison result; If the first comparison result is yes and the second comparison result is yes, the process returns to S204 to continue traversing the next current target file; otherwise, the process executes S208.
5. A method for uploading a WEB static file according to claim 1, characterized in that: The method of comparing and analyzing the source directory and the target directory based on the Diff-utils algorithm to obtain a first data set and a second data set, and judging based on the second data set that if both the source data and the target data are empty, executing S214, and otherwise executing S212, includes: Storing the source directory and the target directory in a cache of a computer; Performing data cleaning on the source files corresponding to the source directory and the target files corresponding to the target directory based on the trim method, wherein the data cleaning at least includes removing blank data; Based on the Diff-utils algorithm, the source files in the source directory are compared and analyzed line by line with the corresponding target files in the target directory to obtain the same line data and the different line data; Constructing a first data set based on each of the same row data, and constructing a second data set based on each of the different row data; Judging the source data and the target data in the second data set, and outputting corresponding marking information according to the judging result; Determine whether to construct a comparison report according to the marking information.
6. A method for uploading a WEB static file according to claim 5, characterized in that: The querying of the second data set to obtain data whose source content and target content are not empty is recorded in the third data set, a comparison report is output based on the third data set in combination with a preset comparison report template, and a manual operation confirmation is performed. If confirmed, S214 is executed, otherwise S206 is executed, including: Performing a filtering query on the second data set to obtain data in which both the source data and the target data are not empty; Storing the data obtained by querying that both the source data and the target data are not empty into a third data set; Obtaining a preset comparison report template, and filling the data in the third data set into the comparison report template to obtain a comparison report; A manual confirmation operation is performed on the comparison report. If the confirmation is successful, the comparison report is output and then S214 is executed; otherwise, S206 is executed.
7. A method for uploading a WEB static file according to claim 1, characterized in that: The uploading of the current target file and transmitting the current target file and the corresponding target directory to the operation end includes: Upload the current target file and determine the directory corresponding to the current target file; Record the current target file and the directory corresponding to the current target file; The recorded current target file and the corresponding directory are published to the corresponding operation terminal according to a preset format.
8. A WEB static file uploading device, applicable to a WEB static file uploading method as claimed in any one of claims 1 to 7, characterized in that: include: The directory construction module is used to construct the correspondence between the source directory and the target directory and traverse the source directory and the target directory based on the correspondence to generate a corresponding List directory tree; The target file checking module is used to determine whether the current target file exists in the List directory tree. If so, S206 is executed; otherwise, S214 is executed. A file status judgment module is used to judge whether the modification time of the source directory is greater than the modification time of the target directory and determine whether the file size of the source file is consistent with that of the current target file. If the modification time of the source directory is greater than the modification time of the target directory and the size of the source file is consistent with that of the current target file, S204 is executed to traverse the next current target file, otherwise S208 is executed; A file type determination module is used to determine whether the file type is a text type, if so, execute S210, if not, add a time suffix to the current target file and then execute S214; A file comparison module, for comparing and analyzing the source directory and the target directory based on the Diff-utils algorithm to obtain a first data set and a second data set, and for judging based on the second data set if both the source data and the target data are empty, then executing S214, otherwise executing S212; The difference data processing module is used to query the second data set to obtain data whose source content and target content are not empty and record them in the third data set, output a comparison report based on the third data set combined with a preset comparison report template, and perform manual operation confirmation. If confirmed, execute S214, otherwise execute S206; The file upload module is used to upload the current target file and transfer the target file and the corresponding target directory to the operation end.
9. An electronic device, comprising a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, and the memory and the processor communicate through the communication bus and the communication interface, characterized in that: When the processor executes the computer program, it implements the instructions of the WEB static file uploading method described in any one of claims 1 to 7.
10. A readable storage medium storing program instructions, when the program instructions are read and executed by a computing device, the computing device executes the WEB static file uploading method according to any one of claims 1 to 7.
Citation Information
Cited By
Item similarity-based SCA detection method, apparatus and device, and medium
CN119690511A
Data synchronization method and device, electronic equipment and storage medium
CN119884056A
