A front-end data compression method, device, equipment, medium and product

Abstract

The application discloses a front-end data compression method, device, equipment, medium and product. The method comprises the following steps: a main thread receives service data, and when a preset compression triggering condition is met, the main thread dynamically converts a pre-embedded script code string into an executable sub-thread, and the script code string comprises compression logic of a lightweight compression engine; the main thread encodes the service data and stores the service data in a binary buffer, and transfers the memory ownership of the binary buffer to the sub-thread through a message communication interface; the sub-thread creates a typed array view based on the binary buffer, calls the lightweight compression engine to compress the typed array view, generates a compression result, and transfers the memory ownership of the compression result to the main thread; and the main thread reports the compression result or locally stores the compression result. The method can realize efficient data compression in the front end, ensure smooth running of a front-end page, and reduce memory occupation and transmission delay.

Description

Technical Field

[0001] This invention relates to the field of data compression technology, and in particular to a front-end data compression method, apparatus, device, medium, and product. Background Technology

[0002] In the mobile internet era, as the business logic carried by the front end becomes increasingly complex, the amount of data that the front end needs to process and report (such as user behavior logs, performance monitoring indicators, page interaction trajectories, offline cached content, etc.) is growing explosively. In order to reduce network bandwidth consumption and improve transmission efficiency, it is usually necessary to compress the data on the front end before reporting or storing it.

[0003] Currently known web front-end data compression solutions include: the browser's native CompressionStream API, which supports data compression but has limited compatibility in older browsers and relatively limited functionality; traditional compression libraries like Pako.js, which are powerful but large in size (approximately 45.6KB after compression), and can cause page lag by blocking UI rendering when run directly on the main thread; and backend compression solutions, which transmit raw data in uncompressed format to the backend for compression and storage by the server, but cannot solve the bandwidth pressure when the frontend reports data. These solutions have the following main drawbacks: main thread blocking: traditional compression libraries occupy the main thread for extended periods while performing compression calculations, leading to UI response delays; excessive library size: introducing traditional compression libraries significantly increases the front-end package size and prolongs the first-screen loading time; complex asynchronous collaboration: using compression libraries in worker threads typically requires separate script file deployment and complex message passing logic design, resulting in high development and maintenance costs; and insufficient dynamism: existing solutions struggle to dynamically adjust compression strategies based on data type or network environment and embed them into existing workflows in real time. Summary of the Invention

[0004] This invention provides a front-end data compression method, apparatus, device, medium, and product to solve the technical problems of main thread blocking, excessively large compression library size, and complex asynchronous collaboration in existing front-end compression solutions.

[0005] According to one aspect of the present invention, a front-end data compression method is provided, comprising: The main thread receives business data. When the preset compression trigger conditions are met, the main thread dynamically converts the pre-embedded script code string into an executable sub-thread. The script code string includes the compression logic of the lightweight compression engine. The main thread encodes the business data and stores it in a binary buffer, then transfers the memory ownership of the binary buffer to the child thread through a message communication interface. The sub-thread creates a typed array view based on the binary buffer, calls the lightweight compression engine to compress the typed array view, generates a compression result, and transfers the memory ownership of the compression result to the main thread; The main thread reports the compression results or stores them locally.

[0006] According to another aspect of the present invention, a front-end data compression apparatus is provided, comprising: The sub-thread creation module is used by the main thread to receive business data. When the preset compression trigger conditions are met, the main thread dynamically converts the pre-embedded script code string into an executable sub-thread. The script code string includes the compression logic of the lightweight compression engine. The data permission transfer module is used by the main thread to encode the business data and store it in a binary buffer, and then transfer the memory ownership of the binary buffer to the child thread through a message communication interface. The compression module is used by the sub-thread to create a typed array view based on the binary buffer, call the lightweight compression engine to compress the typed array view, generate a compression result, and transfer the memory ownership of the compression result to the main thread; The result processing module is used by the main thread to report the compression result or store it locally.

[0007] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the front-end data compression method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the front-end data compression method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the front-end data compression method according to any embodiment of the present invention.

[0010] According to another aspect of the present invention, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the front-end data compression method as described in any embodiment of the present invention.

[0011] This invention dynamically converts pre-embedded script code strings into executable sub-threads via the main thread. These sub-threads then perform compression calculations in the background, avoiding the compression process blocking UI rendering on the main thread. By transferring ownership of the memory in the binary buffer storing business data from the main program to the sub-thread, and vice versa, through a message communication interface, bidirectional zero-copy transmission is achieved, avoiding memory copying overhead during data transfer. Simultaneously, the sub-thread creates a typed array view based on the binary buffer for the lightweight compression engine to read directly, further reducing memory allocation. This invention achieves efficient data compression while ensuring smooth front-end page operation, significantly reducing memory usage and transmission latency.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of a front-end data compression method provided in an embodiment of the present invention; Figure 2 This is a flowchart of a front-end data compression method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a front-end data compression device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the front-end data compression method of this invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] Figure 1 This is a flowchart of a front-end data compression method provided in an embodiment of the present invention. This embodiment is applicable to application scenarios such as front-end log reporting, offline data caching, data transmission in weak network environments, and preprocessing of large file uploads. The method can be executed by a front-end data compression device, which can be implemented in hardware and / or software and can be configured in an electronic device with corresponding data processing capabilities. Figure 1 As shown, the method includes: S110. The main thread receives business data. When the preset compression trigger conditions are met, the main thread dynamically converts the pre-embedded script code string into an executable sub-thread. The script code string includes the compression logic of the lightweight compression engine.

[0018] After receiving business data, the main thread determines whether the preset compression trigger conditions are met. Optionally, the preset compression trigger conditions include at least one of the following: the amount of business data exceeds a set threshold, the number of business data entries exceeds a batch processing threshold, and the current network environment is in a weak network state. A weak network state refers to a communication environment where network transmission quality is below normal levels, specifically including one or more of the following situations: the currently available bandwidth is lower than the minimum bandwidth threshold required for business data transmission, such as 2G, 3G mobile networks, or 4G, 5G networks in signal coverage edge areas; network round-trip latency exceeds a set threshold, resulting in slow data request response; packet loss rate exceeds the normal range during transmission, requiring frequent retransmissions and significantly reducing actual transmission efficiency; or the operating system or browser has enabled data saving mode, such as low data mode on mobile devices or data saving mode on browsers, indicating that the user is currently in a network environment where data traffic is expected to be reduced.

[0019] When the compression trigger condition is met, the main thread dynamically converts the pre-embedded script code string into an executable sub-thread. This script code string includes the compression logic of a lightweight compression engine, which is a compression library based on pure scripts and supporting tree-shaking optimization, preferably fflate (fast flate, a JavaScript compression and decompression library). fflate has undergone architectural optimizations compared to traditional compression libraries: it decouples the compression state from global variables to local variables, facilitating faster memory reclamation by the JavaScript engine; and with the export method of ECMAScript modules (the official modular standard for JavaScript), it supports build tools to import functional modules on demand through tree-shaking optimization, keeping the package size of the core compression functional modules within 10KB. Furthermore, fflate has optimized the LZ77 matching hash algorithm and memory operations, resulting in an overall compression performance improvement of approximately 50% compared to traditional compression libraries such as Pako.js. Through these optimizations, fflate achieves smaller package sizes and faster compression speeds while maintaining a high compression ratio, making it particularly suitable for use in web front-end environments.

[0020] The script code string is pre-embedded in the main thread code and injected by the packaging tool during the build phase; it does not need to be loaded from outside at runtime. If the child thread creation fails, the main thread automatically falls back to the main thread synchronous compression mode and outputs a warning message to the console.

[0021] S120: The main thread encodes the business data and stores it in a binary buffer. Then, it transfers the memory ownership of the binary buffer to the child thread through the message communication interface.

[0022] The binary buffer can be of type ArrayBuffer, and its size is dynamically allocated based on the amount of business data. When the amount of business data exceeds the preset capacity of the buffer, the main thread automatically expands the buffer or processes it in batches using a block-based approach.

[0023] After the main thread completes the creation and initialization of the child thread, it encodes the business data to be compressed and stores it in a binary buffer. For text-type business data, the main thread calls the text encoding interface to convert it to binary format; for business data that is already binary, it is directly stored in the binary buffer. The main thread transfers ownership of the binary buffer's memory to the child thread through a message communication interface. This transfer process does not involve copying memory data; it is a zero-copy transfer method. The main thread completely transfers control of the underlying memory to the child thread. The main thread itself loses access to this memory, while the child thread gains access and can independently read and write to it without synchronization with the main thread. The physical location of the memory itself remains unchanged; what changes is the ownership of the access rights to that memory.

[0024] Optionally, the main thread transfers the memory ownership of the binary buffer to the child thread through the message communication interface, including: when the main thread calls the message communication interface, it declares the binary buffer as a transferable object in the transmission parameter list.

[0025] When the main thread calls the message communication interface (postMessage method), it declares the binary buffer as a transferable object in the transmission parameter list, so that the browser underlying layer transfers the memory ownership of the binary buffer to the child thread.

[0026] By declaring the binary buffer as a transferable object in the transmission parameter list when the main thread calls the message communication interface, the memory ownership of the binary buffer is transferred from the main thread to the child thread. This transfer process does not involve copying memory data; it is a zero-copy transmission method. That is, the main thread completely transfers control of the underlying memory to the child thread, and the main thread itself loses access to that memory. For large-volume business data (such as log batches exceeding 1 megabyte), zero-copy transmission can reduce memory usage and transmission latency by approximately 50% compared to conventional message passing (which requires copying a data copy).

[0027] S130. The sub-thread creates a typed array view based on the binary buffer, calls the lightweight compression engine to compress the typed array view, generates the compression result, and transfers the memory ownership of the compression result to the main thread.

[0028] After receiving memory control of the binary buffer transferred from the main thread, the child thread needs to convert the buffer into a data format readable by the compression engine. Specifically, the child thread calls the typed array constructor with the binary buffer as a parameter to create a typed array view (e.g., an unsigned 8-bit integer array Uint8Array) on the buffer. This typed array view shares the same memory with the binary buffer and does not generate new memory allocations. Through this typed array view, the child thread can read and write data in the binary buffer byte by byte, which can then be read and compressed by the lightweight compression engine.

[0029] After compression, the child thread returns the compressed result to the main thread via a message communication interface. Similar to when the main thread sends business network data, the child thread also uses a zero-copy transmission method when returning the compressed result. In the transmission parameter list, the binary buffer corresponding to the compressed result is declared as a transferable object, and the memory ownership of the binary buffer corresponding to the compressed result is transferred from the child thread to the main thread. The child thread can continue processing the next batch of data as needed or be terminated and its memory reclaimed.

[0030] Through a bidirectional zero-copy transmission mechanism, data transfer between the main thread and child threads during the entire compression process does not involve memory copying operations. For large-volume business data (such as log batches exceeding 1 megabyte), this mechanism significantly reduces memory usage and transmission latency compared to conventional message passing (which requires copying a data copy), effectively reducing memory usage and transmission latency in large-scale data compression scenarios.

[0031] S140, The main thread reports the compression results or stores them locally.

[0032] After receiving the compression result returned by the child thread, the main thread reports the compression result of the business data or stores it locally, depending on the business scenario.

[0033] Optionally, the main thread may report or store the compression result locally, including: when the business scenario is real-time reporting, the main thread may use the compression result as the request body of a network request and call the network request interface to send it to the server; when the business scenario is local persistence, the main thread may write the compression result to the local database; when the business scenario is real-time transmission, the main thread may push the compression result to other clients through a real-time communication interface.

[0034] When the business scenario is real-time reporting (such as real-time reporting of front-end monitoring logs), the main thread uses the compression result as the request body of a network request and calls the network request interface to send it to the server; when the business scenario is local persistence (such as local caching of offline data), the main thread writes the compression result to the local database; when the business scenario is real-time transmission, the main thread pushes the compression result to other clients through the real-time communication interface.

[0035] This invention dynamically converts pre-embedded script code strings into executable sub-threads via the main thread. These sub-threads then perform compression calculations in the background, avoiding the compression process blocking UI rendering on the main thread. By transferring ownership of the memory in the binary buffer storing business data from the main program to the sub-thread, and vice versa, through a message communication interface, bidirectional zero-copy transmission is achieved, avoiding memory copying overhead during data transfer. Simultaneously, the sub-thread creates a typed array view based on the binary buffer for the lightweight compression engine to read directly, further reducing memory allocation. This invention achieves efficient data compression while ensuring smooth front-end page operation, significantly reducing memory usage and transmission latency.

[0036] Figure 2 This is a flowchart of a front-end data compression method provided in an embodiment of the present invention. This embodiment is an optimization and improvement based on the above embodiment. Figure 2 As shown, the method includes: S210. The main thread receives business data. When the preset compression trigger condition is met, the main thread calls the binary object constructor to wrap the script code string into a binary object in memory. The script code string includes the compression logic of the lightweight compression engine.

[0037] S220: The main thread calls the Uniform Resource Locator (URL) creation interface to generate a temporary access address for the binary object.

[0038] S230. The main thread calls the worker thread constructor with the temporary access address as a parameter, instantiates the child thread, and causes the child thread to load and execute the script code pointed to by the temporary access address.

[0039] The main thread calls the binary object constructor (new Blob()) to wrap the core engine code containing the lightweight compression engine's compression logic and the script code string containing the custom compression scheduling logic into a binary object in memory, specifying the content type as script. The main thread then calls the Uniform Resource Locator (URL.createObjectURL()) interface to generate a temporary Uniform Resource Locator pointing to the binary object's memory address, serving as the binary object's temporary access address. Finally, the main thread calls the worker thread constructor (new Worker()) with this temporary access address as a parameter, instantiating a child thread that automatically loads and executes the script code pointed to by the temporary access address. This dynamic injection mechanism eliminates the need to deploy separate script files on the server; all compression logic is dynamically generated and loaded at runtime.

[0040] By using binary object constructors and Uniform Resource Locators (URLs) to create interfaces, pre-embedded script code strings are dynamically converted into executable child threads. Compared to traditional solutions that require separate deployment of Worker (worker thread) script files, this achieves zero deployment of compression logic. The script code strings are pre-embedded in the main thread code, and all compression logic is packaged and released along with the main thread code, eliminating the need to deploy separate Worker script files on the server. This simplifies engineering configuration and deployment processes. It also eliminates additional network requests. Traditional solutions require loading external script files when instantiating a Worker, generating additional network requests. However, by using binary object URL mapping technology to map the script code in memory to a temporary access address, the child thread can directly access the script code from memory. The script loading process generates no network requests, avoiding sub-thread startup anomalies caused by network latency or script loading failures. It supports runtime dynamic decision-making; since the script code string is concatenated and transformed at runtime, the main thread can dynamically decide whether to create a sub-thread, choose a compression strategy, and even dynamically adjust the injected compression logic code based on business data characteristics (such as data size, data type, and current network conditions). This provides flexibility and adaptability not found in traditional static script solutions. Resource management and release are simplified; after the sub-thread finishes use, the main thread can call the Uniform Resource Locator (URL) release interface to destroy temporary access addresses and terminate the sub-thread as needed. Binary objects in memory are then garbage collected, avoiding script file residue and memory leaks.

[0041] S240: The main thread encodes the business data and stores it in a binary buffer. Then, it transfers the memory ownership of the binary buffer to the child thread through the message communication interface.

[0042] S250: The sub-thread creates a typed array view based on the binary buffer, calls the lightweight compression engine to compress the typed array view, generates the compression result, and transfers the memory ownership of the compression result to the main thread.

[0043] Optionally, before the child thread calls the lightweight compression engine to compress the typed array view, the process further includes: the child thread determining the target compression format based on the data type of the business data. Specifically, the child thread parses the data type of the business data, which includes text, log, and binary file types. Based on the identified data type, the child thread determines the target compression format from the compression formats supported by the lightweight compression engine. The target compression format includes either a fast compression format or a high compression ratio format. The fast compression format prioritizes compression speed and is suitable for scenarios with high real-time requirements; the high compression ratio format prioritizes compression ratio and is suitable for scenarios sensitive to bandwidth or storage space.

[0044] By using a sub-thread to adaptively select the compression format based on the data type of the business data, high compression ratio formats can be selected for highly repetitive data such as text and logs, maximizing the compression ratio and significantly reducing data transmission volume and storage space. For data that already has a certain degree of compression, such as binary files, fast compression formats can be selected to avoid unnecessary compression computation overhead and improve overall processing efficiency. This achieves an adaptive match between compression strategies and data characteristics, striking an optimal balance between compression efficiency and compression speed.

[0045] Optionally, the main thread determines the compression mode parameters based on how the business data is generated and passes them to the child thread, so that the child thread calls the compression interface corresponding to the lightweight compression engine to perform compression based on the compression mode parameters; wherein, the compression mode parameters include synchronous mode or streaming mode.

[0046] The preferred lightweight compression engine is fflate, whose core compression interface accepts a typed array view as input. The synchronous compression interface is suitable for complete data blocks obtained at once, returning the complete compression result upon call. The streaming compression interface is suitable for continuously generated streaming data, supporting block-by-block input, outputting a portion of the compressed data immediately after each block is input, and outputting the remaining data upon receiving an end marker, without waiting for all data to be collected. The sub-thread selects the corresponding compression interface based on the compression mode parameters sent by the main thread: when the compression mode parameter is synchronous, it indicates that the data to be compressed is a complete data block obtained at once; the sub-thread calls the synchronous compression interface, compressing the typed array view as input in one go and outputting the compression result. When the compression mode parameter is streaming, it indicates that the data to be compressed is continuously generated streaming data (such as real-time monitoring metric streams); the sub-thread calls the streaming compression interface, pushing the typed array view into the compression stream block by block, and outputting the complete compression result upon receiving an end marker.

[0047] During compression, the compression level and memory allocation strategy can be dynamically adjusted based on data characteristics to achieve an optimal balance between compression ratio and compression speed. Sub-threads can also dynamically adjust the compression level based on the current CPU utilization during compression: lowering the compression level when CPU utilization exceeds 80% and raising the compression level when CPU utilization is below 30%.

[0048] The main thread determines the compression mode parameters based on the generation method of the business data and passes them to the child thread. The child thread then selects the corresponding compression interface, enabling flexible switching between synchronous compression and streaming compression modes. When the business data is a complete block of data obtained all at once (such as a single file uploaded by a user), the synchronous compression interface can complete the compression in one go, resulting in high processing efficiency. When the business data is continuously generated streaming data (such as real-time log streams), the streaming compression interface can push the compression stream block by block, allowing compression to begin without waiting for all data to be collected, thus reducing peak memory usage. It can adaptively match the compression requirements of different business scenarios, optimizing memory usage strategies while ensuring compression efficiency.

[0049] S260, the main thread reports the compression results or stores them locally.

[0050] This invention, through the main thread calling the binary object constructor, the Uniform Resource Locator (URL) creation interface, and the worker thread constructor, dynamically converts pre-embedded script code strings into executable child threads. This achieves zero deployment and runtime dynamic injection of compression logic, eliminating the need for separate deployment of child thread script files and avoiding additional network requests. By transferring ownership of the memory in the binary buffer storing business data from the main thread to the child thread, and vice versa, through a message communication interface, it achieves bidirectional zero-copy transmission, avoiding memory copying overhead during data transfer. Simultaneously, the child thread creates a typed array view based on the binary buffer for the lightweight compression engine to read directly, further reducing memory allocation. This invention achieves efficient data compression while ensuring smooth front-end page operation, significantly reducing memory usage and transmission latency, and simplifying engineering configuration and deployment processes.

[0051] Figure 3 This is a schematic diagram of the structure of a front-end data compression device provided in an embodiment of the present invention. Figure 3 As shown, the device includes: The sub-thread creation module 310 is used for the main thread to receive business data. When the preset compression trigger condition is met, the main thread dynamically converts the pre-embedded script code string into an executable sub-thread. The script code string includes the compression logic of the lightweight compression engine. The data permission transfer module 320 is used by the main thread to encode the business data and store it in a binary buffer, and then transfer the memory ownership of the binary buffer to the child thread through a message communication interface. Compression module 330 is used by the sub-thread to create a typed array view based on the binary buffer, call the lightweight compression engine to compress the typed array view, generate a compression result, and transfer the memory ownership of the compression result to the main thread; The result processing module 340 is used by the main thread to report the compression result or store it locally.

[0052] The front-end data compression device provided in the embodiments of the present invention can execute the front-end data compression method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0053] Optionally, a sub-thread creation module is specifically used for the main thread to call the binary object constructor to wrap the script code string into a binary object in memory; the main thread to call the Uniform Resource Locator (URL) creation interface to generate a temporary access address for the binary object; and the main thread to call the worker thread constructor with the temporary access address as a parameter to instantiate the sub-thread, so that the sub-thread loads and executes the script code pointed to by the temporary access address.

[0054] Optionally, a data permission transfer module is specifically used to declare the binary buffer as a transferable object in the transmission parameter list when the main thread calls the message communication interface.

[0055] Optionally, it also includes a compression mode determination module, used by the main thread to determine the compression mode parameters according to the way the business data is generated and pass them to the child thread, so that the child thread calls the compression interface corresponding to the lightweight compression engine to perform compression according to the compression mode parameters; wherein, the compression mode parameters include synchronous mode or streaming mode.

[0056] Optionally, the preset compression triggering conditions include at least one of the following: the amount of business data exceeds a set threshold, the number of business data entries exceeds a batch processing threshold, and the current network environment type is a weak network state.

[0057] Optionally, the result processing module is specifically used to: when the business scenario is real-time reporting, the main thread uses the compressed result as the request body of a network request and calls the network request interface to send it to the server; when the business scenario is local persistence, the main thread writes the compressed result to the local database; when the business scenario is real-time transmission, the main thread pushes the compressed result to other clients through a real-time communication interface.

[0058] The front-end data compression device described in further detail can also execute the front-end data compression method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0059] According to embodiments of the present invention, the present invention also provides an electronic device, a readable storage medium, and a computer program product.

[0060] Figure 4A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0061] like Figure 4 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory 42 or a random access memory 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 42 or loaded from storage unit 48 into the random access memory 43. The random access memory 43 may also store various programs and data required for the operation of the electronic device 40. The processor 41, read-only memory 42, and random access memory 43 are interconnected via a bus 44. An input / output interface 45 is also connected to the bus 44.

[0062] Multiple components in electronic device 40 are connected to input / output interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0063] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as front-end data compression methods.

[0064] In some embodiments, the front-end data compression method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 40 via read-only memory 42 and / or communication unit 49. When the computer program is loaded into random access memory 43 and executed by processor 41, one or more steps of the front-end data compression method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the front-end data compression method by any other suitable means (e.g., by means of firmware).

[0065] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), payload programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0066] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0067] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0068] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube, liquid crystal display, or monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0069] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0070] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system to address the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0071] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0072] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A front-end data compression method, characterized in that, The method includes: The main thread receives business data. When the preset compression trigger conditions are met, the main thread dynamically converts the pre-embedded script code string into an executable sub-thread. The script code string includes the compression logic of the lightweight compression engine. The main thread encodes the business data and stores it in a binary buffer, then transfers the memory ownership of the binary buffer to the child thread through a message communication interface. The sub-thread creates a typed array view based on the binary buffer, calls the lightweight compression engine to compress the typed array view, generates a compression result, and transfers the memory ownership of the compression result to the main thread; The main thread reports the compression results or stores them locally.

2. The method according to claim 1, characterized in that, The main thread dynamically converts the pre-embedded script code string into an executable child thread, including: The main thread calls the binary object constructor to wrap the script code string into a binary object in memory; The main thread calls the Uniform Resource Locator (URL) creation interface to generate a temporary access address for the binary object. The main thread calls the worker thread constructor with the temporary access address as a parameter to instantiate a child thread, so that the child thread loads and executes the script code pointed to by the temporary access address.

3. The method according to claim 1, characterized in that, The main thread transfers ownership of the binary buffer's memory to the child thread via a message communication interface, including: When the main thread calls the message communication interface, it declares the binary buffer as a transferable object in the transmission parameter list.

4. The method according to claim 1, characterized in that, The method further includes: the main thread determining compression mode parameters based on the way the business data is generated and passing them to the child thread, so that the child thread calls the compression interface corresponding to the lightweight compression engine to perform compression based on the compression mode parameters; wherein, the compression mode parameters include synchronous mode or streaming mode.

5. The method according to claim 1, characterized in that, The preset compression triggering conditions include at least one of the following: the amount of business data exceeds a set threshold, the number of business data entries exceeds a batch processing threshold, and the current network environment is in a weak network state.

6. The method according to claim 1, characterized in that, The main thread reports or stores the compression result locally, including: When the business scenario is real-time reporting, the main thread uses the compressed result as the request body of a network request and calls the network request interface to send it to the server. When the business scenario requires local persistence, the main thread writes the compression result to the local database; When the business scenario involves real-time transmission, the main thread pushes the compression result to other clients through a real-time communication interface.

7. A front-end data compression device, characterized in that, The device includes: The sub-thread creation module is used by the main thread to receive business data. When the preset compression trigger conditions are met, the main thread dynamically converts the pre-embedded script code string into an executable sub-thread. The script code string includes the compression logic of the lightweight compression engine. The data permission transfer module is used by the main thread to encode the business data and store it in a binary buffer, and then transfer the memory ownership of the binary buffer to the child thread through a message communication interface. The compression module is used by the sub-thread to create a typed array view based on the binary buffer, call the lightweight compression engine to compress the typed array view, generate a compression result, and transfer the memory ownership of the compression result to the main thread; The result processing module is used by the main thread to report the compression result or store it locally.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the front-end data compression method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the front-end data compression method according to any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the front-end data compression method according to any one of claims 1-6.

Images