Data tool set generation method and device, equipment and storage medium

Abstract

The invention provides a data tool set generation method and device, equipment and a storage medium. The method comprises the following steps: acquiring an initial source code of a data tool for compression and decompression processing; carrying out compatible patch implantation processing on the initial source code to obtain a compatible code adapted to a plurality of platforms; compiling the compatibility code to obtain a data tool module capable of running through the virtual machine and a corresponding connection layer module; performing adaptive patch implantation processing on the connection layer module to obtain an adaptive layer module adaptive to a plurality of platforms; compiling the data tool module and the adaptation layer module to obtain data tools adapted to the plurality of platforms; and integrating the data tools matched with the plurality of platforms into a data tool set. According to the invention, the efficiency and security of data processing can be improved.

Description

Technical Field

[0001] This application relates to data compression and decompression technologies, and particularly to a method, apparatus, device, and storage medium for generating a data toolset. Background Art

[0002] In the related art, when decompressing a compressed package through a mini-program or a browser, one solution is to transfer the compressed package to a cloud service provider for decompression processing, or transfer the compressed package to the server side for decompression processing. Since it takes a long time to wait when importing a large compressed package, and there is a risk of privacy leakage, resulting in the problem that the content of the compressed package is tampered with; to avoid the above problems, another solution in the related art is to use a JavaScript library of a scripting programming language to achieve compression or decompression in a mini-program or a browser. However, due to the limitation of the compatibility of the JavaScript library, the generality is not good.

[0003] In summary, the related art lacks an effective solution for compression and decompression on multiple platforms including mini-programs and browsers. Summary of the Invention

[0004] Embodiments of this application provide a method, apparatus, device, and storage medium for generating a data toolset, which can enable the compressed and decompressed data tools to run on multiple platforms and implement the functions of compression and decompression at least in a mini-program or a browser.

[0005] The technical solution of the embodiments of this application is implemented as follows:

[0006] Embodiments of this application provide a method for generating a data toolset, the method includes:

[0007] Obtain the initial source code of a data tool for compression and decompression processing;

[0008] Perform compatibility patch implantation processing on the initial source code to obtain compatibility code adapted to multiple platforms, where the multiple platforms include a mini-program platform and a browser platform;

[0009] Perform compilation processing on the compatibility code to obtain a data tool module that can be run through a virtual machine and a corresponding connection layer module;

[0010] Perform adaptation patch implantation processing on the connection layer module to obtain an adaptation layer module adapted to multiple platforms;

[0011] Perform compilation processing on the data tool module and the adaptation layer module to obtain a data tool adapted to the multiple platforms;

[0012] Integrate the data tools adapted to the multiple platforms into a data toolset.

[0013] An embodiment of the present application provides a data processing method for a data tool set, and the method includes:

[0014] Obtain data to be processed, where the data to be processed includes at least one of a compressed package file and a file to be compressed;

[0015] Input the data to be processed into the data tool set;

[0016] Input command line parameters into the data tool set;

[0017] Based on the command line parameters, process the data to be processed through the data tool set to obtain a processing result.

[0018] An embodiment of the present application provides a generation device for a data tool set, including:

[0019] An acquisition module, configured to acquire initial source code of a data tool for compressing and decompressing data;

[0020] A compatibility patch implantation processing module, configured to perform compatibility patch implantation processing on the initial source code to obtain compatibility code adapted to multiple platforms, where the multiple platforms include a mini-program platform and a browser platform;

[0021] A compilation module, configured to perform compilation processing on the compatibility code to obtain a data tool module and a connection layer module that can run through a virtual machine;

[0022] An adaptation patch implantation processing module, configured to perform adaptation patch implantation processing on the connection layer module to obtain an adaptation layer module adapted to multiple platforms;

[0023] The compilation module is further configured to perform compilation processing on the data tool module and the adaptation layer module to obtain a data tool adapted to the multiple platforms;

[0024] An integration module, configured to integrate the data tools adapted to the multiple platforms into a data tool set.

[0025] An embodiment of the present application provides a data processing device for a data tool set, including:

[0026] An acquisition module, configured to acquire data to be processed, where the data to be processed includes at least one of a compressed package file and a file to be compressed;

[0027] An input module, configured to input the data to be processed into the data tool set;

[0028] The input module is further configured to input command line parameters into the data tool set;

[0029] A processing module, configured to process the data to be processed through the data tool set based on the command line parameters to obtain a processing result.

[0030] An embodiment of the present application provides a computer device, which includes:

[0031] A memory, configured to store computer-executable instructions;

[0032] A processor, when executing the computer-executable instructions stored in the memory, implements the data tool set generation method or the data processing method of the data tool set provided by the embodiment of the present application.

[0033] An embodiment of the present application provides a computer-readable storage medium, storing a computer program or computer-executable instructions, which when executed by a processor, implement the data tool set generation method or the data processing method of the data tool set provided by the embodiment of the present application.

[0034] An embodiment of the present application provides a computer program product, including a computer program or computer-executable instructions, which when executed by a processor, implement the data tool set generation method or the data processing method of the data tool set provided by the embodiment of the present application.

[0035] The embodiment of the present application has the following beneficial effects:

[0036] By implanting compatibility patches into the initial source code of the compressed and decompressed data tools, the obtained compatibility code can be compatible with multiple platforms including the applet platform and the browser platform; by implanting adaptation patches into the connection layer module, the data tool module can be docked on the applet platform and the browser platform; then, by compiling the data tool module and the patched connection layer module, a data tool capable of decompressing data on the applet platform and the browser platform is obtained and integrated as a data tool set; through the data tool set, at least the compression and decompression functions in the applet and the browser can be realized, which has better compatibility compared with the related technology of compressing and decoding through a JavaScript library in the applet or the browser; compared with the related technology of cloud compression and decompression, the data security is improved. Description of the Drawings

[0037] Figure 1 It is a schematic structural diagram of the development system of the data tool set provided by the embodiment of the present application;

[0038] Figure 2A It is a schematic structural diagram of the server provided by the embodiment of the present application;

[0039] Figure 2BIt is another schematic diagram of the server structure provided by the embodiments of the present application;

[0040] Figure 3 It is a schematic diagram of the terminal structure provided by the embodiments of the present application;

[0041] Figure 4A It is the first flowchart of the method for generating a data toolset provided by the embodiments of the present application;

[0042] Figure 4B It is the second flowchart of the method for generating a data toolset provided by the embodiments of the present application;

[0043] Figure 4C It is the third flowchart of the method for generating a data toolset provided by the embodiments of the present application;

[0044] Figure 4D It is the fourth flowchart of the method for generating a data toolset provided by the embodiments of the present application;

[0045] Figure 4E It is the fifth flowchart of the method for generating a data toolset provided by the embodiments of the present application;

[0046] Figure 4F It is the sixth flowchart of the method for generating a data toolset provided by the embodiments of the present application;

[0047] Figure 4G It is the seventh flowchart of the method for generating a data toolset provided by the embodiments of the present application;

[0048] Figure 4H It is the eighth flowchart of the method for generating a data toolset provided by the embodiments of the present application;

[0049] Figure 5A It is the first flowchart of the data processing method of the data toolset provided by the embodiments of the present application;

[0050] Figure 5B It is the second flowchart of the data processing method of the data toolset provided by the embodiments of the present application;

[0051] Figure 5C It is the third flowchart of the data processing method of the data toolset provided by the embodiments of the present application;

[0052] Figure 5D It is the fourth flowchart of the data processing method of the data toolset provided by the embodiments of the present application;

[0053] Figure 6 It is a flowchart of the decompression of the compressed package provided by the embodiments of the present application;

[0054] Figure 7AIt is the first operation schematic diagram after decompressing the compressed package provided by the embodiment of the present application;

[0055] Figure 7B It is the second operation schematic diagram after decompressing the compressed package provided by the embodiment of the present application;

[0056] Figure 7C It is the third operation schematic diagram after decompressing the compressed package provided by the embodiment of the present application;

[0057] Figure 7D It is the fourth operation schematic diagram after decompressing the compressed package provided by the embodiment of the present application;

[0058] Figure 7E It is the fifth operation schematic diagram after decompressing the compressed package provided by the embodiment of the present application;

[0059] Figure 8 It is the flow schematic diagram of the generation and use method of the data tool set provided by the embodiment of the present application. Detailed implementation manners

[0060] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0061] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0062] In the following description, the terms "first / second / third" are only used to distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first / second / third" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0063] In the embodiments of the present application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit that includes the function of the module or unit.

[0064] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used in the embodiments of this application are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0065] Before further elaborating on the embodiments of this application, the nouns and terms involved in the embodiments of this application are described. The nouns and terms involved in the embodiments of this application are subject to the following explanations.

[0066] 1. 7-Zip: An open-source compression and decompression software developed using the C and C++ programming languages. It officially supports decompressing dozens of archive formats such as zip, 7z, and RAR, and also supports compressing formats such as zip and 7z. The official version can only run on the three major desktop operating systems: Windows, Mac, and Linux.

[0067] 2. ECMAScript: The specification standard of the JavaScript programming language, which defines various details of the JavaScript programming language. Among them, ECMAScript 6 is the sixth version of ECMAScript and has been supported by mainstream browsers since 2015.

[0068] 3. Browser: Any modern browser after 2015 that supports the ECMAScript 6 standard.

[0069] 4. Mini Program: A program that depends on the browser environment, developed based on web technologies, and can run in the browser environment without installation after being downloaded. For example, it can run in any application (such as an instant messaging client) integrated with a browser environment, or directly run in a browser. Mini programs developed by different companies are basically compatible at the development interface level, enabling the same mini program code to run on multiple platforms.

[0070] 5. Compiled Language: A programming language that can be compiled into machine code and directly uses the resources of the target machine's Central Processing Unit (CPU) for calculation. This type of programming language needs to be compiled in advance for the target machine to run. Commonly used compiled languages include C, C++, Golang, Rust, and Fortran. 7-Zip is developed using C and C++. Corresponding to compiled languages is interpreted languages, which are interpreted line by line during runtime. Since interpreted languages do not directly use the machine code and instructions of the target machine for calculation, their efficiency is lower than that of compiled languages.

[0071] 6. WebAssembly: A technology that enables software developed in compiled languages to run on browsers. It is a virtual instruction set architecture (virtual ISA) that can enhance the performance of browser applications, address the consumption of processor and memory resources by large-scale applications written in JavaScript, and enable many existing high-performance classic software to run in browsers (browser applications or applications integrated with browsers, such as instant messaging clients) after certain modifications to the existing code, without the need for users to manually install, and significantly improve the development efficiency of Web applications. WebAssembly runs in the browser using a virtual machine approach, ensuring performance while having a higher security level, effectively avoiding common security vulnerabilities such as buffer overflows that could cause WebAssembly code to access memory outside of WebAssembly.

[0072] 6. JS-based syntax standard (asm.js): A subset of JavaScript that can be used as a low-level, efficient compiler target language. asm.js provides an abstract implementation similar to a C / C++ virtual machine, including a large binary heap for payload and storage, integer and floating-point operations, higher-order function definitions, function pointers, etc. For browsers that do not support WebAssembly, asm.js provides a WebAssembly fallback solution with performance consistent with JavaScript and good compatibility, still having the ability to port compiled language software like WebAssembly.

[0073] 7. Emscripten: A compiler software that can compile compiled language projects into WebAssembly or asm.js code, supporting most mainstream compiled languages.

[0074] 8. Portable Operating System Interface (POSIX), a set of common interfaces that allow software to be portable between different operating systems without having to be completely rewritten. Linux has a certain POSIX compatibility and supports most POSIX interfaces.

[0075] 9. Glue Code, also called bonding code or glue layer code. The software module implemented by glue code is referred to as glue layer or connection layer in the following text. The purpose of glue code is to connect two incompatible codes or components. For example, C++ and Java can be connected, and the generated code contains both C++ files and Java files. Glue code can be written in the same language as the connected code or component, or in a separate glue language. Glue code uses existing libraries or programs to interoperate in external function interfaces (such as Java native interfaces). Glue code can be a piece of code, a script file, or even a new component.

[0076] In the related art, when compressing or decompressing a compressed package file, the following schemes are included:

[0077] Solution 1: Use existing JavaScript libraries to achieve local compression or decompression in the browser. JSZip is an open source JavaScript library. It is a relatively mature and stable Zip format file compression and decompression library for the Web platform. It is implemented using pure JavaScript so that it can be used in browser scenarios without installing third-party software. It is compatible with all mainstream browsers and is often used in scenarios such as package download and batch import. Developers can introduce the JSZip library into their own projects to call its compression and decompression capabilities and achieve local decompression.

[0078] Here, on the one hand, the JSZip library has compatibility issues and cannot completely replace the mature software of the existing technology. Since the Zip format itself does not store file name encoding information, the file name encoding of the compressed package created by the Windows and Mac platforms is different, and garbled characters will appear in JSZip; JSZip cannot handle compressed packages with passwords and has poor compatibility. In addition, the JSZip library currently only implements the processing of the Zip file format, which cannot meet the wider experience of the desktop. On the other hand, the JSZip library has performance issues. Since JSZip is a library written in JavaScript, and JavaScript is an interpreted language, its performance is worse than that of compiled languages. In the test of zip file decompression, JSZip's performance is only about 10% of 7-Zip.

[0079] Solution 2: Pass the compressed package to the cloud service provider for processing to achieve online decompression. Public cloud service providers all support object storage, which can store files in the cloud and perform operations such as transcoding, auditing, encryption, compression, and decompression on files. Developers need to upload the files to be compressed to the object storage first, then call the decompression interface to obtain the file list, and then call the decompression interface again to decompress all or part of the files in the compressed package.

[0080] The existing problem is that, on the one hand, it is necessary to first upload the local compressed package file to the object storage. If the user's network is poor or the upload speed is slow, it will take a long time to wait, and the decompression status cannot be obtained when calling the decompression interface to decompress the file, resulting in an unsatisfactory overall user experience. On the other hand, in scenarios involving user privacy, certain privacy issues will be caused. It is necessary to transfer the user's compressed package password to the cloud platform, and for files that the user does not want to decompress, the cloud service provider still has the ability to read them, which may lead to the leakage of the user's personal information.

[0081] Solution 3: Transfer the compressed package to the server side for processing to achieve online decompression. Add functions related to server-side decompression to the server-side code of the actual business. The user first uploads the file to the server side, and the server calls existing decompression software (such as WinRAR, 7-Zip, etc.) to read the file and decompress all or part of the files in the compressed package.

[0082] Here, on the one hand, it requires a lot of server resources. Uploading the compressed package file consumes bandwidth and storage resources, and decompression consumes CPU and memory resources. Moreover, the more users there are, the more resources will be consumed. When there are suddenly many users, it is easy to cause server-side congestion or even crashes, affecting the user experience. On the other hand, for some files that the user does not want to decompress, the server-side decompression solution also has the risk of privacy leakage. In addition, for some malicious constructs, especially compressed packages designed for specific versions of decompression software, decompressing on the server side may cause the server to be attacked, resulting in privacy leakage or content tampering of other users.

[0083] Based on the above analysis, it can be determined that the solutions in the related technologies have the following problems:

[0084] 1. Slow speed. The methods in the related technologies have certain performance problems. The performance of the JSZip solution is limited by the performance of the JavaScript programming language, and the solutions of uploading to the cloud service provider or the server are limited by the network performance of the user side. These solutions will cause users to wait for a long time when importing large compressed packages.

[0085] 2. Poor compatibility. For the JSZip solution, its implementation of the Zip format is quite limited, resulting in problems such as garbled characters after decompressing some Zip files, or being unable to decompress due to format incompatibility.

[0086] 3. There are privacy issues. For the solution of uploading to cloud service providers or servers, users do not want some files to be decompressed, and there is also a way to read these files for the solution itself. If the cloud service provider is untrusted or the server is attacked, there will be a risk of privacy leakage for these files, which cannot meet the compliance policies and the protection of users' personal information, and also cannot meet the user needs for some offline scenarios.

[0087] 4. There are security risks. For the solution of uploading to the server, malicious compressed packages can cause the server to be attacked, further leading to the privacy leakage of other users or the content being tampered with.

[0088] To solve the above problems, the embodiments of the present application provide a method, device, equipment and storage medium for generating a data tool set, which can improve the efficiency when decompressing compressed packages, improve the compatibility of data tools, and enhance the security of data processing.

[0089] See Figure 1 , Figure 1 which is a schematic diagram of the architecture of the development system of the data tool set provided by the embodiments of the present application. Figure 1 It involves a data tool set 100, a server 200, a network 300 and a terminal 400. Exemplarily, the terminal 400 can be a smart phone. The terminal 400 can be connected to the server 200 and the data tool set 100 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two. Among them, the data tool set can be developed through the server for the terminal to download and then use in the mini-program platform or browser platform. Or, download the existing data tool set in other browsers and then directly use it in the server platform. The server 200 uses the data processing functions in the data tool set 100 to perform compression and decompression processing on the data to be processed in the terminal 400, including: the server 200 performs compression and decompression on the data to be processed in the server platform through the data tool in the data tool set 100 adapted to the server platform, and the terminal 400 performs compression and decompression on the data to be processed in the mini-program platform and browser platform through the data tool in the data tool set 100 adapted to the mini-program platform and browser platform, so as to realize the compression and decompression processing of data on multiple local platforms.

[0090] In some embodiments, developers can generate a data tool set through the computing resources of the terminal. The terminal can use the data tools in the data tool set adapted to the mini-program platform and the data tools adapted to the browser platform. It can also upload the data tool set to the server, such as the background server of the application store, for users to download and use.

[0091] In some embodiments, the terminal 400 may be implemented as various types of user terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a mobile device (e.g., a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device), a smart phone, a smart speaker, a smart watch, a smart TV, a vehicle terminal, etc.

[0092] In some embodiments, the server 200 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Among them, the cloud service may be an interactive processing service for the terminal to call.

[0093] In some embodiments, multiple servers may form a blockchain, and the server 200 is a node on the blockchain. Each server is used to implement different steps of generating a data toolset. There may be an information connection between each node in the blockchain, and information can be transmitted between nodes through the above information connection. Among them, the data related to the method for generating the data toolset or the data processing method of the data toolset provided in the embodiments of the present application can be stored on the blockchain.

[0094] Taking the computer device for data processing based on the data toolset as the server above as an example, see Figure 2A , Figure 2A which is a schematic structural diagram of the server provided in the embodiments of the present application. Figure 2A The shown server 200-1 includes: at least one processor 210-1, a memory 230-1, and at least one network interface 220-1. Each component in the server 200-1 is coupled together through a bus system 240-1. It can be understood that the bus system 240-1 is used to realize the connection and communication between these components. The bus system 240-1 includes, in addition to a data bus, a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 2A all kinds of buses are labeled as the bus system 240-1.

[0095] The processor 210-1 may be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a Digital Signal Processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0096] The memory 230-1 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disc drives, etc. The memory 230-1 optionally includes one or more storage devices that are physically remote from the processor 210-1.

[0097] The memory 230-1 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 230-1 described in the embodiments of the present application is intended to include any suitable type of memory.

[0098] In some embodiments, the memory 230-1 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which will be described exemplarily below.

[0099] The operating system 231-1, including system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks;

[0100] The network communication module 232-1 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 220-1. Exemplary network interfaces 220-1 include: Bluetooth, Wi-Fi (Wireless Fidelity), and Universal Serial Bus (USB), etc.

[0101] In some embodiments, the device provided by the embodiments of the present application can be implemented in software. Figure 2A The generation device 233 of the data toolset stored in the memory 230-1 is shown, which can be software in the form of programs and plugins, etc., and includes the following software modules: an acquisition module 2331, a compatible patch implantation processing module 2332, a compilation module 2333, an adaptation patch implantation processing module 2334, and an integration module 2335. These modules are logical, so they can be combined arbitrarily or further split according to the functions to be implemented. The functions of each module will be described below.

[0102] Taking the computer device for data processing based on the data toolset as the server above as an example, see Figure 2B , Figure 2B This is another schematic structural diagram of the server provided by the embodiments of the present application. Figure 2BThe server 200-2 shown includes: at least one processor 210-2, a memory 230-2, and at least one network interface 220-2. Each component in the server 200-2 is coupled together through a bus system 240-2. It can be understood that the bus system 240-2 is used to implement the connection and communication between these components. In addition to including a data bus, the bus system 240-2 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 2B all kinds of buses are labeled as the bus system 240-2. For the specific descriptions of the processor 210-2 and the memory 230-2, please refer to the above, and will not be elaborated here.

[0103] In some embodiments, the device provided by the embodiments of the present application can be implemented in software. Figure 2B The data processing device 234 of the data tool set stored in the memory 230-2 is shown, which can be software in the form of a program and a plug-in, etc., and includes the following software modules: an acquisition module 2341, an input module 2342, and a processing module 2343. These modules are logical, so they can be combined arbitrarily or further split according to the functions to be implemented. The data tools in the data processing device 234 of the data tool set that are adapted to the server platform can compress and decompress the data to be processed on the server platform. The functions of each module will be described below.

[0104] Taking the computer device that processes data based on the data tool set as the above-mentioned terminal as an example, refer to Figure 3 , Figure 3 which is the schematic structural diagram of the terminal provided by the embodiments of the present application. Figure 3 The terminal 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. Each component in the terminal 400 is coupled together through a bus system 440. It can be understood that the bus system 440 is used to implement the connection and communication between these components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clear illustration, in Figure 3 all kinds of buses are labeled as the bus system 440.

[0105] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0106] The user interface 430 includes one or more output devices 431 enabling the presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components facilitating user input, such as a keyboard, a mouse, a microphone, a touch screen display, a camera, other input buttons, and controls.

[0107] The memory 350 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid state memory, hard disk drives, optical disk drives, etc. The memory 350 optionally includes one or more storage devices physically located remote from the processor 310.

[0108] The memory 350 includes volatile memory or non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 350 described in the embodiments of the present application is intended to include any suitable type of memory.

[0109] In some embodiments, the memory 350 is capable of storing data to support various operations. Examples of such data include programs, modules, and data structures, or subsets or supersets thereof, which are illustratively described below.

[0110] The operating system 351, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0111] The network communication module 352 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 320. Exemplary network interfaces 320 include: Bluetooth, Wi-Fi (Wireless Fidelity), and Universal Serial Bus (USB), etc.;

[0112] The presentation module 353 is used to enable the presentation of information (such as a user interface for operating peripheral devices and displaying content and information) via one or more output devices 331 associated with the user interface 330 (such as a display screen, a speaker, etc.).

[0113] The input processing module 354 is used to collect one or more user inputs or interactions from one of one or more input devices 332 and translate the collected inputs or interactions.

[0114] In some embodiments, the device provided by the embodiments of the present application may be implemented in software. Figure 3 Shown is a data processing device 455 of a data toolset stored in a memory 450, which may be software in the form of a program and plug-ins, etc., including the following software modules: an acquisition module 4551, an input module 4552, and a combination module 4553. These modules are logical, so they can be combined arbitrarily or further split according to the functions implemented. The data tool adapted to the applet platform and the browser platform in the data processing device 455 of the data toolset can compress and decompress the data to be processed on the applet platform and the browser platform. The functions of each module will be described below.

[0115] Next, the generation method of the data toolset provided by the embodiments of the present application will be described in combination with the exemplary applications and implementations of the server and the terminal provided by the embodiments of the present application. Refer to Figure 4A , Figure 4A is the first process schematic diagram of the generation method of the data toolset provided by the embodiments of the present application, which will be described in combination with Figure 4A the steps shown. As described above, the computer device for completing the generation method of the data toolset may be the server or the terminal described above, which will not be repeated below.

[0116] In step 101, the initial source code of the data tool for compression and decompression processing is obtained.

[0117] Exemplarily, the data tool may be 7-zip, and the corresponding initial source code is the initial source code of 7-zip. Among them, in addition to using 7-Zip as the source code for compilation, p7zip (the Linux port version of 7-Zip) with similar functions can also be used as the source code for compilation, which can generate a WebAssembly module with a smaller volume and better POSIX compatibility.

[0118] In step 102, compatibility patch implantation processing is performed on the initial source code to obtain compatibility code adapted to multiple platforms, where the multiple platforms include the applet platform and the browser platform.

[0119] Here, the multiple platforms include: the applet platform (such as, an instant messaging client applet), the browser platform (including modern browsers supporting WebAssembly and old browsers supporting asm.js).

[0120] In some embodiments, by performing patch processing on the code related to the operating system platform in the initial source code, compatibility code adapted to multiple platforms can be obtained. Refer to Figure 4B , Figure 4AThe shown step 102 can be implemented through the following steps 1021 to 1025. Among them, steps 1021 to 1025 have no order and can be adjusted according to the actual situation. The following is a specific description.

[0121] In step 1021, search for the code related to the operating system platform in the initial source code.

[0122] Among them, there is a large amount of compatibility code for specific operating system platforms (such as Windows, Linux, macOS) in the 7-Zip source code. These codes are distinguished according to the platform during compilation using constants and judgment codes (such as ifdef / ifndef). Some of these codes cannot match the constants under the compilation parameters of the compiler Emscripten, resulting in exceptions. Therefore, it is first necessary to search all the code related to the operating system platform in the initial source code.

[0123] In step 1022, when the code is the first judgment code, add the first no-op command before the branch code of the first judgment code to make the branch code empty code.

[0124] Exemplarily, the first judgment code includes code in the form of "#ifdef __APPLE__... #else...". Add the first no-op command "#elif __EMSRIPTEN__" before the branch code "else" of the first judgment code to keep the branch code empty to avoid entering the "#else" logic and causing compilation or running failures.

[0125] In step 1023, when the code is the second judgment code, wrap the second no-op command outside the second judgment code to make the second judgment code empty code.

[0126] Exemplarily, the second judgment code includes code in the form of "#ifndef" or "#ifneq". It is necessary to wrap a layer of no-op command code "#ifndef EMSRIPTEN" outside its code to avoid entering the "#ifndef" or "#ifneq" logic during Emscripten compilation and causing compilation or running failures.

[0127] In step 1024, when the code is configuration file code, introduce the compiler link library in the configuration file code.

[0128] Exemplarily, when introducing the Emscripten link library in a text file (such as a Makefile), it is necessary to use code in the form of "ifdef EMSCRIPTEN...else.." for Emscripten at the code location of the configuration file (LDFLAGS configuration) to add the Emscripten link library (such as LDFLAGS_EMCC) for Emscripten.

[0129] In step 1025, the specified formats and graphical interface functions included in the code are trimmed to obtain compatibility code adapted to multiple platforms.

[0130] Among them, the specified formats include some uncommon compression or archive formats supported by the initial source code, such as bz, xz, iso, cab, etc. The initial source code also supports graphical interface functions. These formats and functions cannot be used in the compilation environment and need to be trimmed.

[0131] In the embodiments of the present application, since there is a large amount of compatibility code for specific operating system platforms in the initial source code, these codes use constants and judgment codes to distinguish according to the platform during compilation, and some of the codes cannot match the constants under the compilation parameters of the compiler, resulting in exceptions. Therefore, patching the initial source code can obtain compatibility code adapted to multiple platforms. In this way, the code of the data tool can be adapted to multiple platforms, thereby improving the compatibility of the data tool set and enhancing the efficiency and security of data processing.

[0132] Continue to refer to Figure 4A , and continue to describe based on step 102 above.

[0133] In step 103, the compatibility code is compiled to obtain a data tool module and a corresponding connection layer module that can be run through a virtual machine.

[0134] Among them, the data tool module that can be run through a virtual machine on the browser platform can be obtained by compiling through the compiler Emscripten. The data tool module includes a Web Assembly module (Web Assembly module) using WebAssembly and a script module (asm.js module) using the scripting language (asm.js), both of which are based on a virtual environment. The corresponding connection layer module includes glue layer code, which is generated when the data tool module is compiled.

[0135] In some embodiments, step 103 can be implemented in the following way: Compile the compatibility code with the first file, the second file, and the third file established to obtain a data tool module and a connection layer module.

[0136] Here, a first file is specified through a compilation tool, so as to compile the compatibility code established with the first file, the second file, and the third file, and obtain a data tool module and a connection layer module.

[0137] In some embodiments, before compiling the compatibility code, the compatibility code can also be configured first, and then the compatibility code is compiled after the configuration is completed. Refer to Figure 4C , in Figure 4A Before step 103 shown, the following steps 1031 to 1033 can be executed, which are specifically described below.

[0138] In step 1031, a first file is created in the directory of the compatibility code, where the first file is used to specify the product file name.

[0139] Exemplarily, the directory of the compatibility code includes: the CPP / 7zip / Bundles / Alone2 directory, and the first file is the makefile.emcc file, which is used to specify the product file name and includes the makefile file of 7-Zip itself. Among them, specifying the product file name includes: setting PROG = 7zz.js in the makefile.emcc file, then 7zz.js is the file name of the product after construction.

[0140] In step 1032, a second file is created in the directory of the compatibility code, where the second file is used to set the program to run according to preset conditions during startup.

[0141] Exemplarily, the directory of the compatibility code includes: the CPP / 7zip / Bundles / Alone2 directory, and the second file is the pre.js file. By setting Module.noInitialRun = true, the main function is avoided from being executed when the code starts, so as to achieve running according to preset conditions (as required).

[0142] In step 1033, a third file is created in the directory of the compatibility code, where the third file is used to export multiple modules in the compilation tool, so that the multiple modules can be called by the compatibility code.

[0143] Among them, taking the compilation tool as Emscripten as an example, multiple modules in the compilation tool include: FS, NODEFS, WORKERFS, callMain modules. By exporting the FS, NODEFS, WORKERFS, callMain modules of Emscripten, they can be called when the compatibility code runs.

[0144] In the embodiments of the present application, by configuring the compatibility code and then compiling the configured compatibility code through a compiler, the software of the data tool can be made to run in a browser with almost no performance loss. In this way, when the user imports a compressed package, the function of unpacking and processing the files in the compressed package completely on the user's local side without uploading the compressed package to the server is realized, thereby improving the efficiency and security of data processing.

[0145] Continue to refer to Figure 4A , in step 104, an adaptation patch implantation process is performed on the connection layer module to obtain an adaptation layer module adapted to multiple platforms.

[0146] In some embodiments, for a variety of different platforms, by performing corresponding patch processing on the code of the connection layer module, an adaptation layer module adapted to different platforms can be obtained. Refer to Figure 4D , Figure 4A The steps shown in step 104 can be implemented through the following steps 1041 to 1045, which are specifically described below.

[0147] Among them, when the platform is a mini-program platform, the following adaptation patch implantation process is performed, including steps 1041 to 1044:

[0148] In step 1041, a mini-program platform patch implantation process is performed on the connection layer module.

[0149] Here, the mini-program platform patch implantation process is used to patch the connection layer module so that the data tool module can run in the mini-program environment.

[0150] In some embodiments, by performing patch processing on the code of the connection layer module, the code of the data tool module can run properly in the mini-program platform. Refer to Figure 4E , Figure 4D The steps shown in step 1041 can be implemented through the following steps 10411 to 10414, which are specifically described below.

[0151] In step 10411, the rewritten network assembly module code is obtained, and the original network assembly module code in the connection layer module is replaced based on the rewritten network assembly module code, so that the connection layer module can call the mini-program module when calling the network assembly module.

[0152] Among them, for the applet platform, patches are applied to the code of the network assembly module (WebAssembly module). Taking the instant messaging client applet as an example, since the instant messaging client does not support WebAssembly objects and only supports WXWebAssembly objects, and the instantiate method therein only accepts the first parameter as the WebAssembly module path, it is necessary to use a network proxy (Proxy) to rewrite WebAssembly to obtain the rewritten network assembly module code, so that when the connection layer module calls the network assembly module (WebAssembly), it can actually call the applet module (WXWebAssembly).

[0153] In step 10412, obtain the rewritten function for reading binary, and replace the original function for reading binary in the connection layer module based on the rewritten function for reading binary, so that the function for reading binary can return the file path name of the network assembly module.

[0154] Exemplarily, the function for reading binary includes: the readBinary function. Developers replace the original function for reading binary in the connection layer module by rewriting the readBinary function, so that the function for reading binary can return the file path name of the network assembly module.

[0155] In step 10413, disable the asynchronous reading function in the connection layer module.

[0156] Exemplarily, the asynchronous reading function can be the readAsync function.

[0157] In step 10414, set the logic of the judgment constant for non-applet platforms in the connection layer module to false, and set the logic of the judgment constant for the applet platform to true.

[0158] Among them, non-applet platforms include: server platforms and browser platforms (such as, ENVIRONMENT_IS_NODE, ENVIRONMENT_IS_SHELL, etc.). Set the logic of the judgment constants for these platforms to false, and set the logic of the judgment constant for the applet platform (such as, ENVIRONMENT_IS_MP) to true.

[0159] In the embodiments of the present application, since the connection layer module is automatically generated by a compiler and includes methods such as starting a data module, setting up a file system, setting input / output, and calling functions. Therefore, the compiled data tool module can be directly used in a browser to call functions by passing command-line arguments, but it cannot be directly run on a mini-program platform. In the embodiments of the present application, a mini-program platform patch implantation process is performed on the connection layer module for the mini-program platform, so that the code of the connection layer module can run normally on the mini-program platform. In this way, the data tool module is realized to run on the mini-program platform, and when the user imports a compressed package, the files in the compressed package can be unzipped and processed within the mini-program, thereby improving the efficiency and security of data processing.

[0160] Continue to refer to Figure 4D , in step 1042, a file system patch implantation process is performed on the connection layer module.

[0161] In some embodiments, by performing input / output patch processing on the connection layer module, the connection layer module can be adapted to multiple platforms. Refer to Figure 4F , Figure 4D The steps 1042 shown can be implemented through the following steps 10421 to 10426, which are specifically described below.

[0162] In step 10421, a virtual file system module adapted to the mini-program platform is created; among them, the virtual file system module includes file operation methods in the connection layer module.

[0163] Exemplarily, the virtual file system module is an MPFS module. The file operation methods in the connection layer module include: 1. Node operations: getattr, setattr, lookup, mknod, rename, symlink, readlink, unlink, rmdir, readdir; 2. File operations: open, close, read, write, llseek, allocate, mmap, msync.

[0164] In step 10422, for each file operation method in the file operation methods, the virtual file system of the mini-program platform is mapped to the temporary directory of the mini-program platform.

[0165] Here, taking the mini-program platform of an instant messaging client mini-program platform as an example, for each file operation method, by actually calling the corresponding synchronous method under the instance of the instant messaging client file management system (wx.getFileSystemManager), the mapping of the virtual file system to the temporary directory of the instant messaging client mini-program is realized.

[0166] In step 10423, obtain the error information in the applet platform and map the error information to the error code list in the virtual file system module of the connection layer module.

[0167] Exemplarily, the virtual file system module of the connection layer module includes the virtual file system module (FS module) of the glue layer of WebAssembly.

[0168] In step 10424, according to the permission bits of the virtual file system module of the connection layer module, convert the permission bits to the permission bits of the virtual file system module of the applet platform.

[0169] Among them, the permission bits of the virtual file system module of the applet platform are the permission bits of the file system manager (FileSystemManager) in the virtual file system module of the applet platform.

[0170] In step 10425, for the temporary read-only directory of the applet platform, disable the write operation on the temporary directory in the virtual file system of the applet platform.

[0171] In step 10426, in the virtual file system module of the connection layer module, mount the virtual root directory to the in-memory file system, mount the data directory to the temporary directory of the file system of the applet platform, and mount the data temporary directory to the temporary read-only directory of the virtual file system of the applet platform.

[0172] Exemplarily, the in-memory file system can be represented as MEMFS, the data directory is " / data directory", and the data temporary directory is " / data-tmp directory".

[0173] In the embodiment of the present application, since the connection layer module comes with some file systems that can be directly used in the browser platform and the server platform, but only the in-memory file system is available for the implementation of these file systems in the applet platform. And only using the in-memory file system will cause too high memory occupancy. Therefore, the embodiment of the present application performs file system patch implantation processing for the applet platform and adapts the applet platform by constructing the virtual file system of the connection layer. In this way, the performance of the data tool in the applet platform is optimized, and the efficiency and security of data processing are improved.

[0174] Continue to refer to Figure 4D , in step 1043, perform input / output patch implantation processing on the connection layer module.

[0175] In step 1044, use the connection layer module after the adaptation patch implantation processing as the adaptation layer module adapted to the applet platform.

[0176] Among them, when the platform is a browser platform or a server platform, step 1044 can be implemented in the following way: perform input / output patch implantation processing on the connection layer module, and use the connection layer module after the input / output patch implantation processing as an adaptation layer module adapted to the browser platform or the server platform.

[0177] In some embodiments, by performing input / output patch processing on the connection layer module, the connection layer module can be adapted to multiple platforms. Refer to Figure 4G , performing input / output patch implantation processing on the connection layer module can be implemented through the following steps 10431 to 10432, which are specifically described below.

[0178] In step 10431, construct a command line buffer by limiting the maximum number of output lines through a variable-length array or a first-in-first-out queue.

[0179] In step 10432, based on the command line buffer, replace the processing mechanism by which the connection layer module redirects the standard output to the console.

[0180] Here, replace the behavior by which the connection layer module redirects the standard output (stdout) to the console (console) by default through the data tool module (WebAssembly) through the command line buffer.

[0181] Continue to refer to Figure 4A , in step 105, perform compilation processing on the data tool module and the adaptation layer module to obtain a data tool adapted to multiple platforms.

[0182] Among them, the data tool adapted to multiple platforms includes: a data tool adapted to the applet platform (applet platform 7-zip), a data tool adapted to the server platform (Node.js platform 7-zip), a data tool adapted to modern browsers (modern server platform 7-zip), and a data tool adapted to old browsers (old browser platform 7-zip).

[0183] In some embodiments, the multiple platforms further include a server platform (Node.js platform); the data tool module includes at least one of the following: a WebAssembly module (WebAssembly module) that supports WebAssembly and a script module (asm.js module) that supports the script language (asm.js). Refer to Figure 4H , Figure 4A The step 105 shown can be implemented through the following steps 1051 to 1053, which are specifically described below.

[0184] In step 1051, perform compilation processing on the WebAssembly module and the connection layer module adapted to the applet platform to obtain a data tool adapted to the applet platform.

[0185] In step 1052, the connection layer modules of the network assembly module and the adaptation server platform are compiled to obtain a data tool adapted to the server platform.

[0186] Among them, by performing at least one of the following steps 1053 and 1054, a data tool adapted to the browser platform is obtained:

[0187] In step 1053, the connection layer modules of the network assembly module and the adaptation browser platform are compiled to obtain a data tool adapted to the browser platform that supports the network assembly language.

[0188] Among them, the browser that supports the network assembly language is a modern browser, so the obtained data tool is adapted to the modern browser platform.

[0189] In step 1054, the script module and the connection layer module of the adaptation browser platform are compiled to obtain a data tool adapted to the browser platform that supports the script language.

[0190] Among them, the browser platform that supports the script language is an old browser that does not support the network assembly language, so the obtained data tool is adapted to the old browser platform.

[0191] In the embodiment of the present application, since there is an old browser that does not support the network assembly language, therefore, the code of the script module is constructed by using a compiler, and a data tool adapted to the browser platform that supports the script language is constructed through the script module. In this way, the data tool set can run in browsers of different versions, increasing the usage range of the data tool set and improving the compatibility of the data tool set.

[0192] Continue to refer to Figure 4A to continue the description of step 105 above.

[0193] In step 106, the data tools adapted to multiple platforms are integrated into a data tool set.

[0194] Among them, by integrating the data tool adapted to the applet platform (such as, 7-zip for the applet platform), the data tool adapted to the server platform (such as, 7-zip for the Node.js platform), and the data tool adapted to the browser (such as, 7-zip for the modern server platform or 7-zip for the old browser platform), a data tool set (such as, the adapted 7-zip module) is obtained.

[0195] In some embodiments, step 106 can be implemented by the following method: integrating the data tool adapted to the applet platform, the data tool adapted to the server platform, and at least one data tool adapted to the browser platform to obtain a data tool set.

[0196] Among them, by integrating data tools adapted to the applet platform (e.g., 7-zip for the applet platform), data tools adapted to the server platform (e.g., 7-zip for the Node.js platform), data tools adapted to modern browsers (e.g., 7-zip for the modern server platform), and / or data tools adapted to old browsers (e.g., 7-zip for the old browser platform), a data tool set (e.g., the adapted 7-zip module) is obtained.

[0197] In the embodiments of the present application, by obtaining the initial source code of the data tool for compression and decompression processing, performing patch implantation processing and compilation processing, data tools adapted to multiple platforms are obtained and integrated into a data tool set. In this way, the data tool set can be adapted to multiple platforms, enabling local decompression and other processing of data on multiple platforms, thereby improving the performance and efficiency of data processing and enhancing data security.

[0198] The following describes a specific example of data processing of the data tool set described above. The electronic device implementing the data processing method of the data tool set in the embodiments of the present application can be a terminal or a server. Among them, the data tools in the data tool set adapted to the applet platform and the browser platform are for use by the terminal, and the data tools adapted to the server platform are for use by the server. Refer to Figure 5A , Figure 5A is the first flowchart of the data processing method of the data tool set provided in the embodiments of the present application, which will be described in conjunction with Figure 5A the steps shown.

[0199] In step 201, the data to be processed is obtained, where the data to be processed includes at least one of a compressed package file and a file to be compressed.

[0200] In step 202, the data to be processed is input into the data tool set.

[0201] Here, by mounting the data to be processed to the virtual file system of the connection layer module of the data tool set, the data to be processed is input into the data tool set in the form of standard input.

[0202] In step 203, the command line parameters are input into the data tool set.

[0203] Here, by constructing command line parameters corresponding to the data to be processed, the command line parameters are input into the data tool set in the form of standard input.

[0204] In some embodiments, the command line parameters are input into the data tool set in the form of a string array. Refer to Figure 5B , Figure 5AThe shown step 203 can be implemented through the following steps 2031 to 2033, which are specifically described below.

[0205] In step 2031, operation information for compressing or decompressing the data to be processed is received.

[0206] In step 2032, the operation information is encoded into command-line parameters.

[0207] In step 2033, by calling a function, a string array representing the command-line parameters is passed into the data toolkit, where the string array is used to input the command-line parameters into the data toolkit in the form of standard input.

[0208] Continue to refer to Figure 5A to continue the description of step 203 above.

[0209] In step 204, based on the command-line parameters, the data to be processed is processed by the data toolkit to obtain a processing result.

[0210] In some embodiments, the data output from the data toolkit is parsed to obtain the final processing result. Refer to Figure 5C , Figure 5A The shown step 204 can be implemented through the following steps 2041 to 2044, which are specifically described below.

[0211] In step 2041, based on the command-line parameters, the data to be processed is compressed or decompressed by the data toolkit to obtain a standard output and a virtual file system output.

[0212] In step 2042, the directory structure in the standard output is constructed into a directory tree through a command-line buffer.

[0213] Among them, the directory structure in the standard output is constructed into a directory tree through a variable-length array in the command-line buffer.

[0214] In step 2043, for the virtual file system output, a file system management program is used to traverse the temporary directory of the applet platform and parse the virtual file system output to obtain a parsing result.

[0215] Here, an implementation method is proposed by taking the applet platform as an example. By reading and parsing the virtual file system output, it is also applicable to other platforms.

[0216] In step 2044, the directory tree and the parsing result are used as the processing result.

[0217] In the embodiments of the present application, by obtaining the data to be processed, inputting the data to be processed and the corresponding command-line parameters into the data tool set, and then processing the data to be processed according to the command-line parameters through the data tool set, a processing result is obtained. In this way, the data tool set that can run on multiple platforms can be used to perform processing such as decompressing data locally on multiple platforms, thereby improving the performance and efficiency of data processing and enhancing data security.

[0218] In some embodiments, after the data to be processed is processed by the data processing tool to obtain the final processing result, the processed data can also be displayed and further processed. Refer to Figure 5D , Figure 5A After step 204 shown in

[0219] In step 205, the directory tree is displayed according to the file hierarchy before the file to be processed is compressed.

[0220] Exemplarily, refer to Figure 6 , Figure 6 which is a schematic flowchart of decompressing a compressed package provided by an embodiment of the present application.

[0221] In step 206, determine the type of the file to be processed, and perform at least one of the operations of previewing, saving, processing, and forwarding on the processing result according to the type of the file to be processed.

[0222] Exemplarily, refer to Figure 7A , which is a schematic diagram of the first operation after decompressing a compressed package provided by an embodiment of the present application. For file types that can be previewed or uploaded for online documents (such as Word / Excel / PPT / PDF, etc.), previewing can be performed, or uploading to an online document for saving; refer to Figure 7B , which is a schematic diagram of the second operation after decompressing a compressed package provided by an embodiment of the present application. For picture types, picture processing can be performed, such as converting pictures to text, converting pictures to tables, converting pictures to PDF, etc.; refer to Figure 7C , which is a schematic diagram of the third operation after decompressing a compressed package provided by an embodiment of the present application. For video types, they can be saved to the mobile phone or forwarded to friends or contacts of an instant messaging client; refer to Figure 7D , which is a schematic diagram of the fourth operation after decompressing a compressed package provided by an embodiment of the present application. For file types that cannot be previewed and processed, they can be forwarded to friends or contacts of an instant messaging client; refer to Figure 7E , which is a schematic diagram of the fifth operation after decompressing a compressed package provided by an embodiment of the present application. For file types that cannot be previewed and forwarded, they can be saved to the mobile phone. The following will be described in detail.

[0223] Combined with the above method, the embodiments of the present application have the following beneficial effects:

[0224] 1. Improve performance. In the embodiments of the present application, through the implantation of compatibility patches, compilation processing, and the implantation of connection layer module adaptation patches for the initial source code of the data tools for compression and decompression, the data processing tools can run at least on the mini-program platform or in the browser with almost no performance loss, thus solving the problem of slow data processing speed in the related art.

[0225] 2. Improve compatibility. Taking the 7-Zip software, a data tool for compression and decompression processing, as an example, the data tool set obtained in the present application has good compatibility with various compression formats and supports functions such as volume splitting, password, multi-threaded compression and decompression. In addition, for old browsers that do not support WebAssembly, in the embodiments of the present application, asm.js code is constructed for use by old browsers, thereby improving the compatibility of the data tool set.

[0226] 3. Improve privacy protection capabilities. In the stages of importing data and decompressing data, the method in the embodiments of the present application can run entirely in the mini-program or browser. The user's compressed package, the possible password of the compressed package, and the files in the compressed package will all be processed locally by the user and will not be uploaded to any platform. The files in the compressed package will only be uploaded to the server for further processing with the user's permission after the user selects the files to be decompressed and the decompression is completed, thereby improving privacy protection capabilities.

[0227] 4. Improve security. The data tool set generated in the embodiments of the present application runs in a sandbox during the actual use stage. With the strong security of the browser and the isolation of resources and memory by the WebAssembly sandbox mechanism, security vulnerabilities that often occur in compression software, such as directory escape attacks, buffer overflow attacks, privilege escalation attacks, and arbitrary code execution attacks, are effectively avoided. In the case where the user uses a malicious compressed package to trigger the above security vulnerabilities, the malicious compressed package will not access the resources of other pages or other processes outside the page, thereby improving the security of data processing.

[0228] Next, the exemplary application of the embodiments of the present application in an actual application scenario will be described. The daily volume of compressed packages transmitted in the instant messaging client reaches more than 300 million. However, currently, after the user receives a compressed package file, it cannot be previewed in a timely manner. On the mobile phone, a third-party APP or mini-program needs to be used for decompression before it can be used. On the computer side, a professional decompression software needs to be downloaded, and the usage process is very complicated. The user's demand for directly decompressing and previewing files in the instant messaging client is very strong.

[0229] Based on this, the embodiments of the present application provide the ability to quickly decompress compressed packages in an instant messaging client. For mainstream compressed package formats (including zip / 7z / rar, etc.), use the "More Open Methods" of the instant messaging client, and select the online document applet to use, as follows.

[0230] See Figure 6 , Figure 6 which is a schematic flowchart of the decompression of the compressed package provided by the embodiments of the present application. As Figure 6 shown, the process of decompressing the compressed package is demonstrated. After decompression, the directory tree of the files can be displayed according to the file hierarchy before compression. The display interface 61 includes: the file 601 to be decompressed and the decompression control 602. In response to clicking the decompression control 602, the file 601 is decompressed, and the display interface 62 is switched to. The display interface 62 includes the applets 603 that can be used to open files, specifically including: online documents, cloud disk software, and office software. In response to clicking at least one of the applets 603 that can be used to open files, the file is decompressed to obtain the display interface 63. The display interface 63 includes: the compressed package being parsed 604. After the file decompression is completed, the display interface 64 is obtained. The display interface 64 includes the directory tree of the file hierarchy obtained after decompression, and the save control 605 for saving the file. In response to clicking the save control 605, the file is saved.

[0231] After decompressing the compressed package, it supports previewing, saving, processing, or forwarding the decompressed files to friends or contacts in the instant messaging client for different types of files. See Figure 7A , for the file types that can be previewed or uploaded for online documents (such as Word / Excel / PPT / PDF, etc.), they can be previewed or uploaded to the online document for saving. Among them, the display interface 71 includes: the preview content of text documents, the prompt message 711, and the function control 712. The display interface 72 includes: the preview content of documents of types such as PPT or PDF, the prompt message 721, and the function control 722. In the display interface 71, after decompression is completed, the prompt message 711 pops up. In response to clicking the function control 712, the decompressed document is edited and recorded. In the display interface 72, after decompression is completed, the prompt message 721 pops up. In response to clicking the function control 722, reading settings, format conversion, or enabling the toolbox for other operations are performed on the decompressed document.

[0232] See Figure 7B, image processing can be performed on image types, such as converting images to text, converting images to tables, converting images to PDF, etc. Among them, the display interface 73 includes: the preview content 731 of the image type document, and the function controls for operating on the image file: convert image to document 732, convert image to table 733, convert image to PDF 734, and more tools 735. In response to clicking on the corresponding function control, the decompressed file is processed accordingly.

[0233] See Figure 7C , for video types, they can be saved to the mobile phone or forwarded to friends on the instant messaging client. Among them, the display interface 74 includes: the video file being downloaded 741, the prompt message 742, and the download control 743. In response to clicking on the download control 743, the file is downloaded, and after the file is downloaded, it switches to the display interface 75. In the display interface 75, it includes: the downloaded video file 751, the prompt message 752, the save control 753, and the forward control 754. In response to clicking on the save control 753, the downloaded file is saved; in response to clicking on the forward control, the downloaded file is forwarded to friends on the instant messaging client.

[0234] See Figure 7D , for other file types that cannot be previewed and processed, they can be forwarded to friends on the instant messaging client. Among them, the display interface 76 includes: the file being downloaded 761, the prompt message 762, and the download control 763. In response to clicking on the download control 763, the file is downloaded, and after the download is completed, it switches to the display interface 77. The display interface 77 includes: the downloaded file 771, the prompt message 772, and the forward control 773. In response to clicking on the forward control 773, the downloaded file is forwarded to friends on the instant messaging client.

[0235] See Figure 7E , for other file types that cannot be previewed and forwarded, they can be saved to the mobile phone. Among them, the display interface 78 includes: the file 781, the prompt message 782, and the save control 783. In response to clicking on the save control 783, the file is saved, and after the saving is completed, it switches to the display interface 79. In the display interface 79, it includes: the save success prompt 791, the open folder control 792, and the return control 793. In response to clicking on the open folder control 792, the folder where the file 781 is saved is opened, and in response to clicking on the return control 793, it returns to the previous operation interface.

[0236] The following describes the generation and usage examples of the data toolset provided in the embodiments of the present application. See Figure 8 , Figure 8 is a schematic flowchart of the method for generating and using the data toolset provided in the embodiments of the present application. The following will be described in conjunction with Figure 8 the steps shown.

[0237] Among them, the development and adaptation stage includes:

[0238] First, obtain the source code 80 of the data tool (corresponding to the initial source code of the aforementioned data tool). Exemplarily, the data tool is 7-Zip, and the source code of the data tool is the 7-Zip code. In addition to using 7-Zip as the source code for compilation, p7zip (a ported version of 7-Zip under Linux), which has similar functions, can also be used as the source code for compilation. Compared with 7-Zip, it can generate a WebAssembly module with a smaller volume and has better POSIX compatibility.

[0239] In step 801, patch the source code of the data tool (corresponding to implanting a compatibility patch into the initial source code as described above).

[0240] Here, modify and adapt the 7-Zip code. The specific method for patching the 7-Zip source code includes:

[0241] In the 7-Zip source code, there is a large amount of compatibility code for specific operating system platforms (Windows, Linux, macOS). These codes are distinguished according to the platform during compilation using constants and ifdef / ifndef judgments. Some of these codes cannot match the constants under the Emscripten compilation parameters, resulting in exceptions. Therefore, when patching the 7-Zip source code, it is first necessary to search for all codes related to the operating system platform in the source code and handle them for Emscripten according to the following judgment rules:

[0242] 1) If the code is in the form of #ifdef __APPLE__... #else... (corresponding to the aforementioned first judgment code), then #elif __EMSRIPTEN__ (corresponding to the aforementioned first empty command) needs to be added before else (corresponding to the branch code of the first judgment code), and keep the branch code empty to avoid entering the logic in else and causing compilation or running failures.

[0243] 2) If the code is in the form of #ifndef or #ifneq (corresponding to the aforementioned second judgment code)), then it needs to be wrapped with #ifndef EMSRIPTEN (corresponding to the aforementioned second empty command) outside the code to avoid entering the #ifndef or #ifneq logic during Emscripten compilation and causing compilation or running failures.

[0244] 3) In the Makefile code (corresponding to the aforementioned configuration file code), it is necessary to introduce the Emscripten link library (corresponding to the aforementioned compiler link library). At the LDFLAGS configuration, use ifdef EMSCRIPTEN... else... to add the Emscripten link library (LDFLAGS_EMCC) for Emscripten.

[0245] 4) The 7-Zip source code supports some uncommon compression or archive formats, such as bz, xz, iso, cab, etc. (corresponding to the aforementioned specified formats). For these uncommon formats, they can be trimmed as needed. The trimming method is: delete the reference to the corresponding format in the Makefile file (7zip.mak) in the code directory where the corresponding format is referenced.

[0246] 5) The 7-Zip source code supports a graphical interface (corresponding to the aforementioned graphical interface function), which cannot be used in the Emscripten environment. For this function, it can be trimmed as needed. The trimming method is: delete the reference to the UI module in the Makefile file (7zip.mak) in the code directory where the corresponding format is referenced.

[0247] In step 802, use the compiler to compile to obtain the glue layer 81 (corresponding to the aforementioned connection layer module), the script module 82, and the web assembly module 83 (corresponding to the aforementioned data tool module).

[0248] Exemplarily, the compiler is Emscripten, the script module is the asm.js module, and the web assembly module is the WebAssembly module. Compile the 7-Zip code into the glue layer, the WebAssembly module, and the asm.js module through Emscripten.

[0249] The specific method of using Emscripten to compile 7-Zip includes:

[0250] After the 7-Zip 21.0 version, native Linux support is provided and it is operated in a command-line manner. The support for Linux is based on POSIX compatibility, and Emscripten is POSIX-compatible. Therefore, 7-Zip can be compiled based on the compilation method of the Linux version of 7-Zip. However, the following configurations are required before compilation:

[0251] 1) Create a makefile.emcc file (corresponding to the aforementioned first file) in the CPP / 7zip / Bundles / Alone2 directory (corresponding to the directory of the aforementioned compatibility code) to specify the product file name and include the makefile of 7-Zip itself. Here, the specification means: set PROG = 7zz.js in the makefile.emcc file, then 7zz.js is the file name of the built product.

[0252] 2) Create a pre.js file (corresponding to the aforementioned second file) in the CPP / 7zip / Bundles / Alone2 directory, and set Module.noInitialRun = true to avoid executing the main function at startup and achieve on-demand running (corresponding to the aforementioned setting that the program runs according to preset conditions during startup).

[0253] 3) Create a post.js file (corresponding to the aforementioned third file) in the CPP / 7zip / Bundles / Alone2 directory, and export the FS, NODEFS, WORKERFS, callMain modules of Emscripten (corresponding to multiple modules in the aforementioned compilation tool) to Module so that they can be called during runtime.

[0254] After the configuration is completed, use the emmake tool (a tool in the compilation tool) to specify the makefile.emcc file, and then 7-Zip can be compiled into a WebAssembly module and an asm.js module, and the corresponding glue layer code is included.

[0255] In step 803, adapt the glue layer for the browser platform, server platform (such as the Node.js environment), and applet environment, including steps 8031 to 8033, as follows.

[0256] In step 8031, patch the glue layer.

[0257] Here, the compiled WebAssembly module can be directly called in the browser by passing command-line arguments to callMain. However, if it is to be run in the applet environment, the glue layer (used to connect the applet environment and the asm.js / webassembly module) needs to be patched. The glue layer is automatically generated by Emscripten and contains methods such as starting WebAssembly, setting the file system, setting input and output, and calling functions. The specific steps for patching in step 8031 are as follows:

[0258] 1) Patch the WebAssembly module for the instant messaging client applet. Since the instant messaging client does not support WebAssembly objects and only supports WXWebAssembly objects, and the instantiate method only accepts the first parameter as the WebAssembly module path, a Proxy is needed to rewrite WebAssembly (obtaining the aforementioned rewritten network assembly module code) so that when the glue layer calls WebAssembly (corresponding to the aforementioned network assembly module), it can actually call WXWebAssembly (corresponding to the aforementioned applet module).

[0259] 2) Patch the readBinary function (corresponding to the aforementioned binary function) for the instant messaging client applet. Since the instantiate method in 1) only supports the first parameter as the module path, the readBinary method needs to be rewritten to return the file path name (corresponding to the file path name of the aforementioned network assembly module), rather than the ArrayBuffer of the file. Among them, the first parameter of the instantiate of the instant messaging client applet is different from that of the browser. The browser accepts ArrayBuffer while the instant messaging client applet accepts the file location path of the WebAssembly module. Therefore, the file path name here refers to the path of the WebAssembly module relative to the glue layer in the applet running environment. The method used to obtain the path here is: traverse upward from the current path until a module named 7zz.wasm is found, parse its relative path, and return it. Path example:.. / .. / wasm / 7zz.wasm.br.

[0260] 3) Disable the readAsync function (corresponding to the aforementioned asynchronous read function) for the instant messaging client applet and directly set it to undefined to avoid errors caused by the existence of the readAsync function, which may lead to the glue layer using network asynchronous loading of WebAssembly resources.

[0261] 4) For the instant messaging client applet, make the judgment constants for other environments (corresponding to the aforementioned non-applet platforms, other environments except applets, such as Node.js, browsers, etc., such as ENVIRONMENT_IS_NODE, ENVIRONMENT_IS_SHELL, etc.) always false to avoid these judgment constants being mis-set to true in the applet environment, resulting in the execution of code that is incompatible with the applet. And add an ENVIRONMENT_IS_MP according to the applet environment. The role here is to ensure that in the applet environment, code related to other environments will not be mis-executed and cause errors. Example: The method automatically generated by Emscripten to judge ENVIRONMENT_IS_NODE is to judge whether the process object exists. However, during the applet development and compilation process, there is also a process object, but we don't have this object when we finally run. Therefore, for the applet environment, these constants need to be set to false to shield the code logic of other environments. The actual execution method is to first judge whether there is a wx object (this object only exists within the applet). If there is, then all other judgment constants are set to false, and ENVIRONMENT_IS_MP is set to true.

[0262] In step 8032, the file system patch.

[0263] The WebAssembly glue layer comes with some file systems that can be used in browser and Node.js environments. However, in the applet environment, only the in-memory file system (MEMFS) is available for the implementation of these file systems. If only the in-memory file system is used, it will cause too high memory occupancy. Therefore, it is necessary to adapt the applet's file system (MPFS) for the applet environment:

[0264] 1) Create a new MPFS module (corresponding to the virtual file system module of the aforementioned applet platform), which contains some necessary file operation methods in the WebAssembly glue layer (corresponding to the file operation methods in the aforementioned connection layer module). These methods are basically the same as the POSIX file operation methods and mainly include the following categories: 1. Node operations: getattr, setattr, look up, mknod, rename, symlink, readlink, unlink, rmdir, readdir; 2. File operations: open, close, read, write, lseek, allocate, mmap, msync.

[0265] 2) For each file operation method, actually call the corresponding synchronous method under the wx.getFileSystemManager() instance to implement the mapping from the virtual file system to the temporary directory of the instant messaging client applet.

[0266] 3) Capture the error of the FileSystemManager of the instant messaging client applet (corresponding to the aforementioned error message), and map the error message to the error code list of the virtual file system module (FS module) of the WebAssembly glue layer.

[0267] 4) Analyze the permission bits of the FS module in the WebAssembly glue layer, and convert each permission bit into the permission bit of the file system manager (FileSystemManager) of the applet platform (incompatible with POSIX).

[0268] 5) For the temporary read-only directory of the instant messaging client applet (such as the selected file), disable the write operation on the temporary directory in the MPFS implementation.

[0269] 6) Mount the virtual root directory to MEMFS in the FS module, mount the / data directory (corresponding to the aforementioned data directory) to the temporary directory of the applet in MPFS, and mount the / data-tmp directory (corresponding to the aforementioned data temporary directory) to the temporary read-only directory in MPFS.

[0270] In step 8033, input and output patches.

[0271] Among them, to facilitate parsing the command-line output, it is necessary to implement a command-line buffer to replace the behavior that WebAssembly (the data tool module in the connection layer module) defaults to redirecting the standard output (stdout) to the console. The command-line buffer can be directly implemented using a variable-length array or a first-in-first-out queue to limit the maximum number of output lines to avoid uncontrolled memory growth.

[0272] Through the above steps 8031 to 8033, the patched glue layer 84 is obtained. Among them, the patched glue layer 84 includes: the glue layer 841 adapted to the applet, the glue layer 842 adapted to the server (such as Node.js), and the glue layer 843 adapted to the browser.

[0273] In some embodiments, the code can be further streamlined on the basis of the patch, directly deleting the unnecessary format support and modules, and it can be achieved that according to different file format scenarios, the 7-Zip module that only supports this file format is dynamically loaded. It is expected that the volume of the compiled WebAssembly module can be minimized to 20% of the current size.

[0274] After that, the adapted code is compiled by the Emscripten compiler, and then the resulting code that can run in mini-programs, Node.js, and browser environments is generated. Specifically, it includes:

[0275] The network assembly module 83 and the glue layer 841 adapted to the mini-program platform are compiled by the compiler to obtain the mini-program platform data tool 851; the network assembly module 83 and the glue layer 842 adapted to the server are compiled by the compiler to obtain the server platform data tool 852; the network assembly module 83 and the glue layer 843 adapted to the browser platform are compiled by the compiler to obtain the modern browser platform data tool 853; the script module 82 and the glue layer 843 of the browser platform are compiled by the compiler to obtain the old browser data tool 854. Then, the mini-program platform data tool 851, the server platform data tool 852, the modern browser platform data tool 853, and the old browser data tool 854 are integrated as the adapted data toolset module 85 (corresponding to the aforementioned data toolset).

[0276] Among them, the running stage includes:

[0277] In step 804, it is mounted to the glue layer virtual file system.

[0278] Among them, the compressed file / zipped file (corresponding to the aforementioned data to be processed) is input into the data toolset in the form of standard input by being mounted to the glue layer virtual file system (the virtual file system of the glue layer in the data toolset). By mounting the selected file as the virtual file system into WebAssembly, the memory overhead is reduced.

[0279] Due to the space occupation limit of the temporary file system of the instant messaging client, in addition to using different mount points, the function of the swap partition can also be implemented in the FS module. The file is split into file blocks and preferentially placed in the temporary directory. If the temporary directory is full, some file blocks can be swapped to the memory file system for buffering. This solution can support files with an uncompressed size exceeding 200M in the instant messaging client mini-program.

[0280] In step 805, the corresponding command line arguments are constructed.

[0281] Among them, the actual compression and decompression operations are encoded as command-line parameters and passed to 7-Zip. The specific construction method of the command-line parameters includes: to pass command-line parameters to 7-Zip, the callMain function (corresponding to the aforementioned call function) needs to be used to pass in an array of strings, where each string represents a command-line parameter, similar to argv in the C language. Taking the example that the user selects test.zip from the instant messaging client chat file in the instant messaging client applet. At this time, calling callMain(['l', ' / data-temp / test.zip']) can view the command-line output of 7-Zip in the console, and the output result is the directory structure of the temporary file. Then, calling callMain(['x', ' / data-temp / test.zip', ' / data']) can use 7-Zip to decompress the compressed package file to the temporary directory of the instant messaging client applet.

[0282] In step 806, through the parser, the standard input / output and file system of the adapted data toolkit module 85 are parsed and processed to obtain the decompression / compression status and results.

[0283] Among them, the specific parsing methods for standard output and file system output include: by default, the standard output will be output to the console, which is the default behavior of the WebAssembly glue layer. However, if it is necessary to read the directory structure or capture error information, the console needs to be redirected. In step 8033, a variable-length array is implemented for stdout. In actual use, this variable-length array needs to be consumed, and the flat directory structure is constructed into a directory tree. For file system output, in the instant messaging client applet environment, the FileSystemManager can be used to traverse the applet's temporary directory and read the output files for parsing or processing. After use, the standard output needs to be cleared, and all output files need to be deleted to save system resources.

[0284] In the embodiments of this application, 7-Zip is compiled into a WebAssembly module, and in particular, the file system of the instant messaging client applet is adapted, which improves the applicability of WebAssembly.

[0285] 1. Performance improvement: The embodiments of this application use highly optimized 7-Zip code, and there is a significant improvement in performance compared with the high-performance WebAssembly runtime. As shown in Table 1, it is the test result of decompressing a 500M Zip file on the same computer:

[0286] Table 1

[0287] Uncompression duration (seconds) Additional memory occupancy 7-Zip 23.01 (Linux) 3.1 3.5M 7-Zip 23.01 (Node.js WebAssembly module) 4.8 16.5M 7-Zip 23.01 (Instant messaging client applet WebAssembly module) 6.5 18.2M 7-Zip 23.01 (Instant messaging client applet asm.js module) 12.3 82.5M JSZip 18.2 1.2G Instant messaging client applet FileSystemManager.unzip 8.1 32.8M Online Zip file uncompression service 11.2 Unable to count

[0288] As can be seen from Table 1, the compiled 7-Zip WebAssembly and asm.js modules outperform JSZip and the FileSystemManager.unzip method of the instant messaging client applet in terms of decompression duration and additional memory occupation. They also have certain advantages compared with cloud services. And due to no need for downloading, they have even greater advantages than those shown in the table compared with cloud services.

[0289] 2. Compatibility improvement: Compared with JSZip, unzip, online cloud file decompression services, etc., the embodiments of this application support not only Zip file format, but also multiple common file formats such as rar and 7zip, and fully support functions such as encrypted compressed packages, solid compression, and verification of compressed package integrity, having the best compatibility.

[0290] In some embodiments, in addition to instant messaging client applets, browsers, and Node.js scenarios, it can also be used in scenarios such as desktop applications packaged by Electron technology, mobile APPs, and mobile web pages.

[0291] The embodiments of this application propose a technology to transplant the 7-Zip software to the Web platform and make it run on applets of mainstream modern browsers, so as to realize the function of completely decompressing and processing the files in the compressed package on the user's local side without uploading the compressed package to the server when the user imports the compressed package. It solves the problem in the related technology that the compressed package must be uploaded to the server before decompressing the compressed package by (tools with online decompression ability), protects user privacy, saves the time for the user to wait for uploading after importing the compressed package, and improves the user experience.

[0292] The following continues to describe the implementation of the data toolset generation device 233 provided by the embodiments of this application as an exemplary structure of software modules. In some embodiments, as Figure 2A shown, the software modules stored in the data toolset generation device 233 in the memory 230-1 may include:

[0293] An acquisition module 2331, configured to acquire the initial source code of a data tool for compressing and decompressing data.

[0294] A compatibility patch implantation processing module 2332, configured to perform compatibility patch implantation processing on the initial source code to obtain compatibility code adapted to multiple platforms, where the multiple platforms include the applet platform and the browser platform.

[0295] A compilation module 2333, configured to perform compilation processing on the compatibility code to obtain a data tool module and a connection layer module that can run through a virtual machine.

[0296] The adaptation patch implantation processing module 2334 is used to perform adaptation patch implantation processing on the connection layer module to obtain an adaptation layer module adapted to multiple platforms.

[0297] The compilation module 2333 is further used to perform compilation processing on the data tool module and the adaptation layer module to obtain data tools adapted to the multiple platforms.

[0298] The integration module 2335 is used to integrate the data tools adapted to the multiple platforms into a data tool set.

[0299] In some embodiments, the compatibility patch implantation processing module 2332 is further used to search for code related to the operating system platform in the initial source code. When the code is the first judgment code, a first no-op command is added before the branch code of the first judgment code to make the branch code an empty code; when the code is the second judgment code, a second no-op command is wrapped outside the second judgment code to make the second judgment code an empty code; when the code is configuration file code, a compiler link library is introduced into the configuration file code; the specified format and graphical interface functions included in the code are trimmed to obtain the compatibility code adapted to the multiple platforms.

[0300] In some embodiments, the compilation module 2333 is further used to create a first file in the directory of the compatibility code, where the first file is used to specify the product file name; create a second file in the directory of the compatibility code, where the second file is used to set the program to run according to preset conditions during startup; create a third file in the directory of the compatibility code, where the third file is used to export multiple modules in the compilation tool so that the multiple modules can be called by the compatibility code; compile the compatibility code with the first file, the second file, and the third file created to obtain the data tool module and the connection layer module.

[0301] In some embodiments, when the platform is a mini-program platform, the adaptation patch implantation processing module 2334 is further used to perform the following adaptation patch implantation processing: perform mini-program platform patch implantation processing on the connection layer module; perform file system patch implantation processing on the connection layer module; perform input / output patch implantation processing on the connection layer module; use the connection layer module after the adaptation patch implantation processing as the adaptation layer module adapted to the mini-program platform; when the platform is a browser platform or a server platform, perform input / output patch implantation processing on the connection layer module, and use the connection layer module after the input / output patch implantation processing as the adaptation layer module adapted to the browser platform or the server platform.

[0302] In some embodiments, the adaptation patch implantation processing module 2334 is further configured to obtain the rewritten network assembly module code, replace the original network assembly module code in the connection layer module based on the rewritten network assembly module code, so that the connection layer module can call the applet module when calling the network assembly module; obtain the rewritten function for reading binary, replace the original function for reading binary in the connection layer module based on the rewritten function for reading binary, so that the function for reading binary can return the file path name of the network assembly module; disable the asynchronous reading function in the connection layer module; set the logic of the judgment constant of the non-applet platform in the connection layer module to false, and set the logic of the judgment constant of the applet platform to true.

[0303] In some embodiments, the adaptation patch implantation processing module 2334 is further configured to create a virtual file system module adapted to the applet platform; wherein, the virtual file system module includes the file operation methods in the connection layer module; for each file operation method in the file operation methods, map the virtual file system of the applet platform to the temporary directory of the applet platform; obtain the error information in the applet platform, and map the error information to the error code list in the virtual file system module of the connection layer module; according to the permission bits of the virtual file system module of the connection layer module, convert the permission bits to the permission bits of the virtual file system module of the applet platform; for the temporary read-only directory of the applet platform, disable the write operation on the temporary directory in the virtual file system of the applet platform; in the virtual file system module of the connection layer module, mount the virtual root directory to the memory file system, mount the data directory to the temporary directory of the file system of the applet platform, and mount the data temporary directory to the temporary read-only directory of the virtual file system of the applet platform.

[0304] In some embodiments, the plurality of platforms further includes a server platform; the data tool module includes at least one of the following: a network assembly module that supports network assembly language and a script module that supports script language; the compilation module 2333 is further configured to compile the network assembly module and the connection layer module adapted to the applet platform to obtain a data tool adapted to the applet platform; compile the network assembly module and the connection layer module adapted to the server platform to obtain a data tool adapted to the server platform; obtain a data tool adapted to the browser platform by performing at least one of the following processes: compile the network assembly module and the connection layer module adapted to the browser platform to obtain a data tool adapted to the browser platform that supports network assembly language; compile the script module and the connection layer module adapted to the browser platform to obtain a data tool adapted to the browser platform that supports script language; the integration module 2331 is further configured to integrate the data tool adapted to the applet platform, the data tool adapted to the server platform, and at least one data tool adapted to the browser platform to obtain the data tool set.

[0305] Next, the exemplary structure of the data processing device 234 of the data tool set in the server 200-2 provided by the embodiments of the present application, which is implemented as a software module, will be further described. In some embodiments, as Figure 2B shown, the data processing device 234 of the data tool set stored in the memory 230-2 may include:

[0306] An acquisition module 2341, configured to acquire data to be processed, where the data to be processed includes at least one of a compressed package file and a file to be compressed.

[0307] An input module 2342, configured to input the data to be processed into the data tool set.

[0308] The input module 2342 is further configured to input the command line parameters into the data tool set.

[0309] A processing module 2343, configured to process the data to be processed through the data tool set based on the command line parameters to obtain a processing result.

[0310] In some embodiments, the input module 2342 is further configured to receive operation information for compressing or decompressing the data to be processed; encode the operation information as command line parameters; and pass in a string array representing the command line parameters in the data tool set by calling a function, where the string array is used to input the command line parameters into the data tool set in the form of standard input.

[0311] In some embodiments, the processing module 2343 is further configured to perform compression or decompression processing on the data to be processed based on the command line parameters through the data tool set, to obtain a standard output and a virtual file system output; construct a directory tree for the directory structure in the standard output through a command line buffer; for the virtual file system output, use a file system management program to traverse a temporary directory of the applet platform and parse the virtual file system output to obtain a parsing result; and use the directory tree and the parsing result as a processing result.

[0312] In some embodiments, the processing module 2343 is further configured to display the directory tree according to the file hierarchy before the file to be processed is compressed; determine the type of the file to be processed, and perform at least one of operations of previewing, saving, processing, and forwarding on the processing result according to the type of the file to be processed.

[0313] Next, an exemplary structure of the data processing device 455 of the data tool set in the terminal 400 provided in the embodiments of the present application as a software module will be further described. In some embodiments, as Figure 3 shown, the data processing device 455 of the data tool set stored in the memory 450 may include:

[0314] An acquisition module 4551, configured to acquire data to be processed, where the data to be processed includes at least one of a compressed package file and a file to be compressed.

[0315] An input module 4552, configured to input the data to be processed into the data tool set.

[0316] The input module 4552 is further configured to input the command line parameters into the data tool set.

[0317] A processing module 4553, configured to perform processing on the data to be processed based on the command line parameters through the data tool set to obtain a processing result.

[0318] In some embodiments, the input module 4552 is further configured to receive operation information for compressing or decompressing the data to be processed; encode the operation information as command line parameters; and pass in a string array representing the command line parameters in the data tool set by calling a function, where the string array is used to input the command line parameters into the data tool set in the form of standard input.

[0319] In some embodiments, the processing module 4553 is further configured to perform compression or decompression processing on the data to be processed based on the command-line parameters through the data toolset, to obtain a standard output and a virtual file system output; construct a directory structure in the standard output into a directory tree through a command-line buffer; for the virtual file system output, traverse a temporary directory of the applet platform using a file system management program, and parse the virtual file system output to obtain a parsing result; and use the directory tree and the parsing result as a processing result.

[0320] In some embodiments, the processing module 4553 is further configured to display the directory tree according to the file hierarchy before the file to be processed is compressed; determine the type of the file to be processed, and perform at least one of operations of previewing, saving, processing, and forwarding on the processing result according to the type of the file to be processed.

[0321] An embodiment of the present application provides a computer program product, which includes a computer program or computer-executable instructions, and the computer program or computer-executable instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer-executable instructions from the computer-readable storage medium, and the processor executes the computer-executable instructions, so that the computer device executes the generation method of the data toolset or the data processing method of the data toolset in the embodiments of the present application.

[0322] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, in which computer-executable instructions or a computer program are stored. When the computer-executable instructions or the computer program are executed by a processor, the processor will be caused to execute the generation method of the data toolset or the data processing method of the data toolset provided in the embodiments of the present application. For example, as Figure 4A shown in the generation method of the data toolset or as Figure 5A shown in the data processing method of the data toolset.

[0323] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0324] In some embodiments, the computer-executable instructions may be in the form of a program, software, software module, script, or code, and may be written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including being deployed as an independent program or being deployed as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0325] As an example, the computer-executable instructions may or may not correspond to files in a file system, and may be stored as part of a file that holds other programs or data. For example, they may be stored in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program under discussion, or in multiple cooperating files (such as files that store one or more modules, subroutines, or portions of code).

[0326] As an example, the computer-executable instructions may be deployed to execute on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0327] In summary, through the embodiments of the present application, a technique for porting a compressed and decompressed data tool software to multiple platforms is proposed, enabling it to run on mini-programs of mainstream modern browsers. When a user imports a compressed package, the function of completely decompressing the files in the compressed package and processing the files locally without uploading the compressed package to the server can be achieved. This solves the problem in the related art that the compressed package must be uploaded to the server before decompressing, saves the time for the user to wait for uploading after importing the compressed package, thereby improving the efficiency and security of data processing and optimizing the user experience.

[0328] The above is only the embodiments of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for generating a data tool set, characterized in that, the method includes: Obtain the initial source code of the data tool for compression and decompression processing; Perform compatible patch implantation processing on the initial source code to obtain compatibility code adapted to multiple platforms, where the multiple platforms include a mini-program platform and a browser platform; Perform compilation processing on the compatibility code to obtain a data tool module that can run through a virtual machine and a corresponding connection layer module; Perform adaptation patch implantation processing on the connection layer module to obtain an adaptation layer module adapted to multiple platforms; Perform compilation processing on the data tool module and the adaptation layer module to obtain a data tool adapted to the multiple platforms; Integrate the data tools adapted to the multiple platforms into a data tool set.

2. The method according to claim 1, characterized in that, the performing compatible patch implantation processing on the initial source code to obtain compatibility code adapted to multiple platforms includes: Search for code related to the operating system platform in the initial source code; When the code is the first judgment code, add a first no-op command before the branch code of the first judgment code to make the branch code an empty code; When the code is the second judgment code, wrap the second judgment code with a second no-op command to make the second judgment code an empty code; When the code is configuration file code, introduce a compiler link library into the configuration file code; Crop the specified format and graphical interface functions included in the code to obtain the compatibility code adapted to multiple platforms.

3. The method according to claim 1, characterized in that, before performing compilation on the compatibility code to obtain a data tool module that can run through a virtual machine and a corresponding connection layer module, the method further includes: Create a first file in the directory of the compatibility code, where the first file is used to specify the product file name; Create a second file in the directory of the compatibility code, where the second file is used to set the program to run according to preset conditions during startup; Create a third file in the directory of the compatibility code, where the third file is used to export multiple modules in the compilation tool so that the multiple modules can be called by the compatibility code; the performing compilation on the compatibility code to obtain a data tool module that can run through a virtual machine and a corresponding connection layer module includes: Perform compilation on the compatibility code with the first file, the second file, and the third file created to obtain the data tool module and the connection layer module.

4. The method according to claim 1, characterized in that, the performing adaptation patch implantation processing on the connection layer module to obtain an adaptation layer module adapted to multiple platforms includes: When the platform is a mini-program platform, perform the following adaptation patch implantation processing: Perform mini-program platform patch implantation processing on the connection layer module; Perform file system patch implantation processing on the connection layer module; Perform input / output patch implantation processing on the connection layer module; Use the connection layer module after the adaptation patch implantation process as the adaptation layer module adapted to the applet platform; When the platform is a browser platform or a server platform, perform input / output patch implantation processing on the connection layer module, and use the connection layer module after the input / output patch implantation process as the adaptation layer module adapted to the browser platform or the server platform.

5. The method according to claim 4, wherein, the performing applet platform patch implantation processing on the connection layer module includes: Obtain the rewritten network assembly module code, and replace the original network assembly module code in the connection layer module based on the rewritten network assembly module code, so that the connection layer module can call the applet module when calling the network assembly module; Obtain the rewritten function for reading binary, and replace the original function for reading binary in the connection layer module based on the rewritten function for reading binary, so that the function for reading binary can return the file path name of the network assembly module; Disable the asynchronous reading function in the connection layer module; Set the logic of the judgment constant for non-applet platforms in the connection layer module to false, and set the logic of the judgment constant for the applet platform to true.

6. The method according to claim 4, wherein, the performing file system patch implantation processing on the connection layer module includes: Create a virtual file system module adapted to the applet platform; wherein, the virtual file system module includes the file operation methods in the connection layer module; For each file operation method in the file operation methods, map the virtual file system of the applet platform to the temporary directory of the applet platform; Obtain the error information in the applet platform, and map the error information to the error code list in the virtual file system module of the connection layer module; According to the permission bits of the virtual file system module of the connection layer module, convert the permission bits to the permission bits of the virtual file system module of the applet platform; For the temporary read-only directory of the applet platform, disable the write operation on the temporary directory in the virtual file system of the applet platform; In the virtual file system module of the connection layer module, mount the virtual root directory to the memory file system, mount the data directory to the temporary directory of the file system of the applet platform, and mount the data temporary directory to the temporary read-only directory of the virtual file system of the applet platform.

7. The method according to claim 4, wherein, the performing input / output patch implantation processing on the connection layer module includes: Construct a command line buffer by limiting the maximum number of output items through a variable-length array or a first-in-first-out queue; Based on the command line buffer, replace the processing mechanism in the connection layer module that redirects the standard output to the console.

8. The method according to any one of claims 1 to 7, wherein, the multiple platforms further include a server platform; the data tool module includes at least one of the following: a network assembly module that supports network assembly language and a script module that supports script language; Compiling the data tool module and the connection layer module adapted to multiple platforms to obtain a data tool adapted to the multiple platforms, including: Compiling the network assembly module and the connection layer module adapted to the applet platform to obtain a data tool adapted to the applet platform; Compiling the network assembly module and the connection layer module adapted to the server platform to obtain a data tool adapted to the server platform; Obtaining a data tool adapted to the browser platform by performing at least one of the following processes: Compiling the network assembly module and the connection layer module adapted to the browser platform to obtain a data tool adapted to a browser platform that supports network assembly language; Compiling the script module and the connection layer module adapted to the browser platform to obtain a data tool adapted to a browser platform that supports script language; Integrating the data tools adapted to the multiple platforms into a data tool set, including: Integrating the data tool adapted to the applet platform, the data tool adapted to the server platform, and at least one data tool adapted to the browser platform to obtain the data tool set.

9. A data processing method for a data tool set, characterized in that, the data tool set is generated by the method according to any one of claims 1 to 8, and the method includes: Obtaining data to be processed, where the data to be processed includes at least one of a compressed package file and a file to be compressed; Inputting the data to be processed into the data tool set; Inputting command line parameters into the data tool set; Based on the command line parameters, processing the data to be processed by the data tool set to obtain a processing result.

10. The method according to claim 9, characterized in that, inputting the command line parameters into the data tool set includes: Receiving operation information for compressing or decompressing the data to be processed; Encoding the operation information as command line parameters; Passing, by calling a function, a string array representing the command line parameters into the data tool set, where the string array is used to input the command line parameters into the data tool set in the form of standard input.

11. The method according to claim 10, characterized in that, processing the data to be processed by the data tool set based on the command line parameters to obtain a processing result includes: Based on the command line parameters, performing compression or decompression processing on the data to be processed by the data tool set to obtain standard output and virtual file system output; Constructing a directory tree for the directory structure in the standard output through command line buffering; For the virtual file system output, traversing the temporary directory of the applet platform using a file system management program and parsing the virtual file system output to obtain a parsing result; Taking the directory tree and the parsing result as the processing result.

12. The method according to any one of claims 9 to 11, characterized in that, After processing the data to be processed through the data tool set to obtain a processing result, the method further includes: Displaying a directory tree according to the file hierarchy before the file to be processed is compressed; Determining the type of the file to be processed, and performing at least one of previewing, saving, processing, and forwarding the processing result according to the type of the file to be processed.

13. A data tool set generation device Characterized in that The device includes: An acquisition module, configured to acquire an initial source code of a data tool for compressing and decompressing data; A compatibility patch implantation processing module, configured to perform compatibility patch implantation processing on the initial source code to obtain compatibility code adapted to multiple platforms, where the multiple platforms include a mini-program platform and a browser platform; A compilation module, configured to perform compilation processing on the compatibility code to obtain a data tool module and a connection layer module that can run through a virtual machine; An adaptation patch implantation processing module, configured to perform adaptation patch implantation processing on the connection layer module to obtain an adaptation layer module adapted to multiple platforms; The compilation module is further configured to perform compilation processing on the data tool module and the adaptation layer module to obtain a data tool adapted to the multiple platforms; An integration module, configured to integrate the data tools adapted to the multiple platforms into a data tool set.

14. A data processing device for a data tool set Including: An acquisition module, configured to acquire data to be processed, where the data to be processed includes at least one of a compressed package file and a file to be compressed; An input module, configured to input the data to be processed into the data tool set; The input module is further configured to input command line parameters into the data tool set; A processing module, configured to process the data to be processed through the data tool set based on the command line parameters to obtain a processing result.

15. A computer device Characterized in that The computer device includes: A memory, configured to store computer executable instructions; A processor, configured to implement the method according to any one of claims 1 to 12 when executing the computer executable instructions stored in the memory.

16. A computer-readable storage medium storing computer executable instructions or a computer program Characterized in that The computer executable instructions or the computer program implement the method according to any one of claims 1 to 12 when executed by a processor.

17. A computer program product including computer executable instructions or a computer program Characterized in that The computer executable instructions or the computer program implement the method according to any one of claims 1 to 12 when executed by a processor.

Images