Big data rendering method and big data rendering system
By requesting data optimization from the server when the browser exceeds the data limit, the browser crash problem caused by exceeding the data limit was solved, and normal rendering was achieved without restricting data query conditions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-14
AI Technical Summary
Existing methods for rendering massive amounts of data often result in excessively long loading times, page lag, or even browser crashes when the data volume exceeds the browser's capacity, and there is a lack of effective solutions.
When the amount of data to be rendered exceeds the maximum amount of data that the browser can render, a data analysis request is sent to the server. The server performs data analysis to determine the target optimization information. The client generates an optimization request. The server processes the data according to the optimization request, obtains the optimized data to be rendered, and returns it to the client for rendering.
By optimizing the data volume to be less than or equal to the maximum data volume that the browser can render, excessively long loading times, page lag, and browser crashes are avoided, ensuring that the browser can render the data normally.
Smart Images
Figure CN121858813A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of big data rendering technology, and in particular to a big data rendering method and a big data rendering system. Background Technology
[0002] With the rapid development of big data technology, it has been widely applied in fields such as public safety, business intelligence, and the Internet of Things. As big data technology becomes more prevalent, the rendering of massive amounts of data has become an indispensable and crucial element in many industry scenarios.
[0003] Current methods for rendering massive amounts of data primarily employ Canvas-based rendering. Specifically, users select query criteria on a browser's visual page, then call a data query service API (Application Programming Interface). The data query server filters the data based on the query criteria and returns it to the browser, which then renders the data using the Canvas API. However, this rendering method suffers from a sharp decline in performance when the data volume exceeds the browser's capacity, leading to excessively long loading times, page stuttering, and even browser crashes.
[0004] Existing big data rendering methods suffer from problems such as excessively long loading times, page lag, and even browser crashes when the data volume exceeds the browser's capacity. Currently, there is no effective solution to these issues. Summary of the Invention
[0005] Therefore, it is necessary to provide a big data rendering method and a big data rendering system to address the aforementioned technical problems.
[0006] Firstly, this application provides a big data rendering method. The method includes:
[0007] When the received data volume of the data to be rendered is greater than the maximum data volume that the browser can render, a data analysis request is sent to the server. The data analysis request is used to instruct the server to perform data analysis on the data to be rendered and determine the target optimization information in the data to be rendered. The target optimization information consists of data optimization items that optimize the data volume of the data to be rendered to be less than or equal to the maximum data volume that the browser can render, and description information corresponding to the data optimization items.
[0008] The server receives the target optimization information returned by the server and generates an optimization request for the data to be rendered based on the user's revision information regarding the target optimization information. The server then sends the optimization request to the server. The optimization request instructs the server to optimize the data to be rendered according to the data optimization items in the optimization request to obtain the optimized data to be rendered.
[0009] Receive the optimized data to be rendered sent by the server, and perform data rendering on the received optimized data to be rendered.
[0010] In one embodiment, before sending a data analysis request to the server when the received data volume to be rendered is greater than the maximum data volume that the browser can render, the process includes:
[0011] Receive the data to be rendered sent by the server and determine the data volume of the received data to be rendered;
[0012] Determine the maximum amount of data that the browser can render;
[0013] Determine whether the received data volume of the data to be rendered is greater than the maximum data volume that the browser can render.
[0014] In one embodiment, determining the maximum amount of data that the browser can render includes:
[0015] Determine the available memory information of the browser;
[0016] Based on the current scene and a preset scene memory mapping table, the memory value occupied by unit data rendering in the current scene is determined; the preset scene memory mapping table is a mapping table between each scene and the memory value occupied by unit data rendering corresponding to each scene.
[0017] Based on the browser's available memory information and the memory value occupied by rendering a unit of data in the current scene, the maximum amount of data that the browser can render is determined.
[0018] In one embodiment, determining the browser's available memory information includes:
[0019] Based on the baseline value of the browser's available memory and the browser's memory coefficient, a recommended value for the browser's available memory is determined;
[0020] The maximum safe value of available memory for the browser was determined using a security isolation test.
[0021] The smaller of the suggested value of the browser's available memory and the maximum safe value of the browser's available memory is determined as the browser's available memory information.
[0022] In one embodiment, determining a suggested value for the browser's available memory based on a baseline value of the browser's available memory and the browser's memory coefficient includes:
[0023] A baseline value for the browser's available memory is determined using either a first calculation method or a second calculation method; the first calculation method is a standard API calculation method; the second calculation method is a calculation method combining the number of CPU cores and UA detection.
[0024] Based on the monitoring results of the browser's JS heap usage, the browser's memory coefficient is determined;
[0025] The recommended value for the browser's available memory is determined by multiplying the baseline value of the browser's available memory by the browser's memory coefficient.
[0026] In one embodiment, determining the browser's memory coefficient based on the monitoring results of the browser's JS heap usage includes:
[0027] Based on the monitoring results of the browser's JS heap usage, data information on the browser's JS heap usage is determined; the data information includes the amount of allocated memory, the amount of memory used, and the amount of memory remaining.
[0028] Calculate a first coefficient and a second coefficient for the browser's memory usage; the first coefficient is the ratio of the used memory to the allocated memory; the second coefficient is the ratio of the remaining memory to the allocated memory.
[0029] Based on the weights of the first coefficient and the second coefficient, the first coefficient and the second coefficient are weighted and summed to obtain the browser's memory coefficient.
[0030] Secondly, this application also provides a big data rendering method. The method includes:
[0031] In response to a received data analysis request from the user, the system performs data analysis on the data to be rendered, determines the target optimization information in the data to be rendered, and returns the target optimization information to the user. The target optimization information consists of data optimization items that optimize the data volume of the data to be rendered to a level less than or equal to the maximum data volume that the browser can render, and description information corresponding to the data optimization items.
[0032] In response to an optimization request received from the user terminal for the data to be rendered, the data to be rendered is optimized according to the data optimization items in the optimization request to obtain optimized data to be rendered, and then sent to the user terminal for rendering; the optimization request is generated by the user terminal based on the revision information received from the user regarding the target optimization information.
[0033] Thirdly, this application also provides a big data rendering method. Applied to a big data rendering system, which includes a user terminal and a server terminal, the method includes:
[0034] The user terminal, in response to the data query conditions selected by the user, generates a data mining instruction for the data to be rendered and sends the generated data mining instruction to the server.
[0035] The server, in response to the received data mining instruction, performs data mining operations to obtain the data to be rendered, and sends the obtained data to be rendered to the user terminal;
[0036] The user terminal receives the data to be rendered, determines the data volume of the received data to be rendered, determines the maximum data volume that the browser can render, determines whether the data volume of the received data to be rendered is greater than the maximum data volume that the browser can render, and when the data volume of the received data to be rendered is greater than the maximum data volume that the browser can render, sends a data analysis request to the server.
[0037] The server, in response to the received data analysis request, performs data analysis on the data to be rendered, determines the target optimization information in the data to be rendered, and returns the target optimization information to the user terminal; the target optimization information consists of data optimization items that optimize the data volume of the data to be rendered to be less than or equal to the maximum data volume that the browser can render, and description information corresponding to the data optimization items.
[0038] The user terminal displays the received target optimization information; and based on the user's revision information of the target optimization information, generates an optimization request for the data to be rendered, and sends the optimization request to the server.
[0039] The server, in response to the received optimization request, optimizes the data to be rendered according to the data optimization items in the optimization request, obtains the optimized data to be rendered, and sends it to the user terminal.
[0040] The user terminal performs data rendering on the received optimized data to be rendered.
[0041] In one embodiment, the server includes a web server and a large model server, and the method includes:
[0042] The web server receives data analysis requests from the user and sends the received data analysis requests to the large model server.
[0043] The large model server, based on the received data analysis request, performs data analysis on the data to be rendered, determines the target optimization information in the data to be rendered, and returns the target optimization information to the WEB server;
[0044] The web server sends the received target optimization information to the user terminal;
[0045] The web server receives an optimization request for the data to be rendered from the user client and sends the received optimization request to the large model server; the optimization request is generated by the user client based on the revision information received from the user regarding the target optimization information;
[0046] The large model server optimizes the data to be rendered based on the data optimization items in the received optimization request, obtains the optimized data to be rendered, and sends it to the WEB server.
[0047] The web server sends the received optimized data to be rendered to the user terminal for rendering.
[0048] Fourthly, this application also provides a big data rendering system. The system includes: a user terminal and a server terminal;
[0049] The user terminal is used to execute the big data rendering method described in the first aspect above;
[0050] The server is used to execute the big data rendering method described in the second aspect above.
[0051] The aforementioned big data rendering method and system, specifically the big data rendering method applied to the user end, determines that the data volume of the data to be rendered exceeds the maximum data volume that the browser can render. It then sends a data analysis request to the server, instructing the server to analyze the data to be rendered, identify target optimization information, receive the target optimization information returned by the server, and generate an optimization request for the data to be rendered based on the user's revision information. This optimization request is then sent to the server, instructing the server to optimize the data to be rendered according to the data optimization items in the optimization request, reducing the data volume to be less than or equal to the browser's renderable data volume. The system obtains optimized data to be rendered from the maximum data size, and receives optimized data from the server that is less than or equal to the maximum data size that the browser can render. It then renders the received optimized data by analyzing the data size to be rendered, obtaining target optimization information that can optimize the data size to be rendered to less than or equal to the maximum data size that the browser can render. This information then instructs the server to optimize the data size to be rendered to less than or equal to the maximum data size that the browser can render, without restricting data query conditions. This ensures that the browser can render the data normally, avoiding excessively long loading times, page lag, or even browser crashes. This solves the problem of excessively long loading times, page lag, or even browser crashes in existing big data rendering methods when the data volume exceeds the browser's capacity limit.
[0052] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description
[0053] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0054] Figure 1 A hardware structure block diagram of a terminal for a big data rendering method provided in an embodiment of this application;
[0055] Figure 2 A flowchart of the big data rendering method provided in Embodiment 1 of this application;
[0056] Figure 3 A flowchart of the big data rendering method provided in Embodiment 2 of this application;
[0057] Figure 4 A flowchart of the big data rendering method provided in Embodiment 3 of this application;
[0058] Figure 5 This is a structural block diagram of a big data rendering system provided in an embodiment of this application. Detailed Implementation
[0059] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.
[0060] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.
[0061] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the big data rendering method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.
[0062] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the big data rendering method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.
[0063] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.
[0064] This first embodiment provides a big data rendering method for the user end. Figure 2 This is a flowchart of the big data rendering method in this embodiment, such as... Figure 2 As shown, the process includes the following steps:
[0065] Step S210: When the received data volume of the data to be rendered is greater than the maximum data volume that the browser can render, a data analysis request is sent to the server. The data analysis request is used to instruct the server to perform data analysis on the data to be rendered and determine the target optimization information in the data to be rendered. The target optimization information is the data optimization item and the description information corresponding to the data optimization item, which optimizes the data volume of the data to be rendered to be less than or equal to the maximum data volume that the browser can render.
[0066] The data volume of the aforementioned data to be rendered can be a quantitative assessment indicator of its impact on browser memory consumption, determined by the data scale and complexity of the data structure. Alternatively, it can simply represent the amount of browser memory consumed; that is, the estimated total system memory usage when the data to be rendered is fully loaded into the client (browser) memory and constructed into a DOM (Document Object Model) or Canvas object for visualization in a specific rendering scenario. It is important to clarify that the aforementioned data volume of the data to be rendered is an estimated performance load, not the actual size of the data during network transmission. The aforementioned DOM is an API (Application Programming Interface) used to render and interact with any HTML (Hypertext Markup Language) or XML (Extensible Markup Language) document. The DOM is a document model loaded into the browser, representing the document as a node tree, where each node represents a component of the document (e.g., page elements, strings, or comments).
[0067] The maximum amount of data that the browser can render can be considered as an estimated threshold for the maximum amount of data memory required for smooth, stable processing without performance issues such as lag or crashes in the current application scenario. This maximum amount of data that the browser can render can be determined based on the browser's available memory and the memory usage per unit of data rendered in the current scenario. The "current scenario" refers to the scenario of the current device environment.
[0068] In this step, the aforementioned data analysis request is an API interface request that includes raw data query condition information, used to instruct the server to perform data analysis on the data to be rendered corresponding to the raw data query condition information. The raw data query condition information can be query information used to obtain the data to be rendered; specifically, it can be data query condition information generated based on the data query conditions selected by the user on the visualization page provided on the user's end. The data query conditions can include information such as time period, entity information, and affinity.
[0069] Step S220: Receive the target optimization information returned by the server, and generate an optimization request for the data to be rendered based on the user's revision information regarding the target optimization information. Send the optimization request to the server. The optimization request instructs the server to optimize the data to be rendered according to the data optimization items in the optimization request, and obtain the optimized data to be rendered.
[0070] The aforementioned user-revision information regarding the target optimization information can be generated by the user revising the data optimization items and their corresponding descriptions in the target optimization information using generalized natural language input. The user-side integrates this revision information into API interface parameters. After the user confirms, an optimization request for the data to be rendered is received and sent to the server. The server optimizes the data to be rendered based on the data optimization items in the optimization request, obtaining the optimized data, and returns it to the user-side. Because the server optimizes the data based on the data optimization items in the optimization request, and these data optimization items are those that can optimize the data size of the data to be rendered to a level less than or equal to the maximum data size that the browser can render, the optimized data size is less than or equal to the maximum data size that the browser can render.
[0071] Step S230: Receive the optimized data to be rendered sent by the server, and perform data rendering on the received optimized data to be rendered.
[0072] Steps S210 to S230 above involve determining that when the received data size of the data to be rendered exceeds the maximum data size that the browser can render, the data size of the data to be rendered exceeds the range that the browser can render normally. A data analysis request is then sent to the server, instructing the server to analyze the data to be rendered based on the data analysis request, determine the target optimization information in the data to be rendered, and then receive the target optimization information returned by the server. Based on the user's revision information regarding the target optimization information, an optimization request for the data to be rendered is generated and sent to the server. The optimization request instructs the server to optimize the data to be rendered according to the data optimization items in the optimization request, optimizing the data size of the data to be rendered to be less than or equal to the maximum data size that the browser can render, thus obtaining the optimized data. The system receives optimized data from the server, which is less than or equal to the maximum data size that the browser can render. It then performs data rendering on this optimized data. By analyzing the data size of the data to be rendered, it obtains optimization information that can optimize the data size of the data to be rendered to less than or equal to the maximum data size that the browser can render. This information then instructs the server to optimize the data size of the data to be rendered to less than or equal to the maximum data size that the browser can render, without restricting data query conditions. This ensures that the browser can render the data normally, avoiding excessively long loading times, page lag, or even browser crashes. This solves the problem of excessively long loading times, page lag, or even browser crashes in existing big data rendering methods when the data volume exceeds the browser's capacity limit.
[0073] In one embodiment, prior to step S210, the following steps are included:
[0074] Step S201: Receive the data to be rendered sent by the server and determine the data volume of the received data to be rendered.
[0075] The determination of the received data volume of the data to be rendered can be achieved by using a preset algorithm model to convert the data size of the received data into a performance load estimate, thus obtaining the data volume of the received data to be rendered. The preset algorithm model can be a fully trained model capable of converting the received data size into the data volume of the data to be rendered. It should be noted that the type and working process of the preset algorithm model can be specifically set based on specific needs. This embodiment does not impose specific limitations, as long as it can determine the data volume of the data to be rendered based on the received data size.
[0076] Preferably, after receiving the data to be rendered sent by the server, the user can also temporarily store the data to be rendered in the client cache.
[0077] Step S202: Determine the maximum amount of data that the browser can render.
[0078] In this step, determining the maximum amount of data that the browser can render can be done by determining the browser's available memory information and, based on the browser's available memory information and the memory value occupied by rendering a unit of data in the current scene, determining the maximum amount of data that the browser can render.
[0079] The memory usage of a unit of data rendering in the current scenario can be the average amount of memory consumed when a single standard data unit (a record, a node, a point, etc.) is fully rendered into the browser view in the current scenario.
[0080] Step S203: Determine whether the data volume of the received data to be rendered is greater than the maximum data volume that the browser can render.
[0081] Steps S201 to S203 above involve receiving the data to be rendered sent by the server, determining the data size of the received data to be rendered, and determining the maximum data size that the browser can render. Then, it is determined whether the data size of the received data to be rendered is greater than the maximum data size that the browser can render. Based on the determination that the data size of the received data to be rendered is greater than the maximum data size that the browser can render, it is determined whether the current browser can render the data to be rendered normally. In order to achieve the goal of optimizing the data to be rendered when the current browser cannot render the data to be rendered normally, the data size of the data to be rendered is optimized to be less than or equal to the maximum data size that the browser can render, so as to achieve normal rendering of the optimized data to be rendered.
[0082] Specifically, in one embodiment, prior to step S201, the following is included:
[0083] Step S200: In response to the data query conditions selected by the user, a data mining instruction for the data to be rendered is generated and sent to the server. The data mining instruction is used to instruct the server to perform data mining operations according to the data query conditions to obtain the data to be rendered.
[0084] To accurately obtain the data to be rendered, users need to select data query conditions on the user-side visualization page. In response to the user's selected query conditions, the user-side interface generates a data mining command that can call the server's web server API interface to retrieve the data to be rendered, and then sends the generated data mining command to the server's web server.
[0085] In another embodiment, step S202, determining the maximum amount of data that the browser can render, includes:
[0086] Step S2022: Determine the browser's available memory information.
[0087] The available memory information of the browser mentioned above represents the maximum remaining memory capacity that the browser process can safely allocate to data rendering tasks without causing performance issues (such as stuttering or crashing) at the current moment and under the current device environment.
[0088] Step S2024: Based on the current scene and the preset scene memory mapping table, determine the memory value occupied by unit data rendering in the current scene; the preset scene memory mapping table is a mapping table between each scene and the memory value occupied by unit data rendering corresponding to each scene.
[0089] Before determining the memory usage per unit of data rendering in the current scene based on the current scene and a pre-defined scene memory mapping table, it is necessary to construct the pre-defined scene memory mapping table. This pre-defined scene memory mapping table can be constructed by obtaining the memory usage per unit of data rendering in a specific scene through persistent stress testing under a stable environment, storing it in the form of a mapping table, and obtaining a table showing the correspondence between each scene and the memory usage per unit of data rendering for each scene.
[0090] The aforementioned preset scene memory mapping table includes scene type, scene name, memory usage per unit of data rendering in different scenes, and the correspondence between these three. Determining the memory usage per unit of data rendering in the current scene based on the current scene and the preset scene memory mapping table can be achieved by retrieving the memory usage per unit of data rendering corresponding to the current scene from the preset scene memory mapping table based on the current scene type and scene name, and then determining the retrieved result as the memory usage per unit of data rendering in the current scene.
[0091] Step S2026: Based on the browser's available memory information and the memory value occupied by rendering a unit of data in the current scene, determine the maximum amount of data that the browser can render.
[0092] The above method, which determines the maximum amount of data that the browser can render based on the browser's available memory information and the memory value occupied by rendering a unit of data in the current scene, can be done by first calculating the difference between the browser's available memory information and the browser's basic memory usage to obtain the actual available memory, and then determining the quotient of the actual available memory and the memory value occupied by rendering a unit of data in the current scene as the maximum amount of data that the browser can render.
[0093] Steps S2022 to S2026 above determine the browser's available memory information, and then, based on the browser's available memory information and the memory value occupied by unit data rendering in the current scene, determine the maximum amount of data that the browser can render. By determining the maximum amount of data that the browser can render, it is easier to determine whether the amount of data received to be rendered is greater than the maximum amount of data that the browser can render, and thus determine whether the current browser can render the data to be rendered normally, and optimize the data to be rendered in a timely manner.
[0094] In one embodiment, step S2022, determining the browser's available memory information, includes:
[0095] Step S1: Based on the baseline value of the browser's available memory and the browser's memory coefficient, determine the recommended value of the browser's available memory.
[0096] The aforementioned recommended value for available browser memory, determined based on a baseline value and a memory coefficient, can be calculated by multiplying the baseline value and the memory coefficient. The baseline value can be an initial estimate of available browser memory obtained directly through standard measurement methods or APIs without any adjustments. The memory coefficient can be a dynamically calculated multi-factor product used to adjust and correct the baseline value to obtain a safer and more reliable recommended value. The recommended value can be the actual available memory value used for rendering decisions after adjustment with the memory coefficient. Therefore, the recommended value for available browser memory represents a safe, reliable, and scenario-appropriate decision value.
[0097] Step S2: Use a security isolation test to determine the maximum safe value of the browser's available memory.
[0098] The aforementioned security isolation test method can be a test method that determines the security boundary (the maximum safe value of the browser's available memory) by gradually increasing memory allocation in a controlled sandbox environment and observing browser performance degradation and failure points.
[0099] Step S3: Determine the smaller of the browser's suggested available memory value and the browser's maximum safe available memory value as the browser's available memory information.
[0100] For example, if the recommended value for available memory in a browser is 1440M and the maximum safe value for available memory in a browser is 2200M, then the available memory information for the browser is 1440M.
[0101] Steps S1 to S3 above determine the suggested value of the browser's available memory, the maximum safe value of the browser's available memory, and the smaller of the suggested value and the maximum safe value of the browser's available memory as the browser's available memory information. This ensures that when rendering is performed based on the determined browser's available memory information, the safe boundary will not be exceeded under any circumstances, and normal rendering can be performed.
[0102] In another embodiment, step S1, determining a suggested value for the browser's available memory based on a baseline value of the browser's available memory and the browser's memory coefficient, includes:
[0103] Step S12: Determine the baseline value of the browser's available memory using either the first calculation method or the second calculation method; the first calculation method is the standard API calculation method; the second calculation method is a calculation method that combines the number of CPU cores and UA detection.
[0104] The standard API calculation method described above can directly obtain memory usage information through the browser's standard APIs and combine it with the total system memory to calculate a baseline value for available memory. Specifically, `performance.memory` (Chrome) or `window.performance.memory` (some browsers) can be used to obtain current memory usage data, including `usedJSHeapSize` (JS heap usage) and `totalJSHeapSize` (total JS heap size). Then, the total device memory can be obtained through system APIs (such as `navigator.deviceMemory`), or the device type can be inferred using `window.screen` and `navigator.userAgent` to estimate the total memory. Finally, the total device memory is subtracted from the current browser memory usage, and then the system's reserved memory is subtracted to obtain a baseline value for the browser's available memory. The calculation method combining CPU core count and UA detection can infer device performance by detecting the device's CPU core count and user agent, and then estimate the baseline value for available memory using preset rules. The detection of the device's CPU core count can be obtained by using `navigator.hardwareConcurrency` to obtain the browser's available logical CPU cores. The aforementioned user agent detection can be achieved by resolving device type (such as mobile phone, tablet, etc.), operating system (such as Android, iOS, Windows, etc.), and browser type (such as Chrome, Safari) using navigator.userAgent. The aforementioned preset rules can be used to classify performance levels based on the number of CPU cores and device type, and then match the corresponding memory baseline values.
[0105] Step S14: Determine the browser's memory coefficient based on the monitoring results of the browser's JS heap usage.
[0106] In this step, the browser's memory coefficient is determined based on the monitoring results of the browser's JS heap usage. This data information, derived from the monitoring results, determines the browser's JS heap usage. Then, a first coefficient and a second coefficient for the browser's memory are calculated. Based on the weights of the first and second coefficients, a weighted sum is performed to obtain the browser's memory coefficient. The first and second coefficients can vary depending on the specific application scenario; this embodiment does not impose specific limitations, as long as the sum of the first and second coefficients is equal to 1.
[0107] Step S16: The product of the baseline value of the browser's available memory and the browser's memory coefficient is determined as the suggested value of the browser's available memory.
[0108] Steps S12 to S16 above determine the baseline value of the browser's available memory through a preset method. Then, based on the monitoring results of the browser's JS heap usage, the browser's memory coefficient is determined. The product of the baseline value of the browser's available memory and the browser's memory coefficient is determined as the suggested value of the browser's available memory. By determining the suggested value of the browser's available memory, it is easier to determine the browser's available memory information based on the suggested value of the browser's available memory.
[0109] Further, in one embodiment, step S14, determining the browser's memory coefficient based on the monitoring results of the browser's JS heap usage, includes:
[0110] Step S142: Based on the monitoring results of the browser's JS (JavaScript Heap) heap usage, determine the data information of the browser's JS heap usage; the data information includes the amount of allocated memory, the amount of memory used, and the amount of memory remaining.
[0111] Before determining the browser's JavaScript heap usage data based on the monitoring results, it is necessary to determine whether the currently accessing browser supports performance.memory. If it does, the browser's JavaScript heap usage data can be determined based on the monitoring results.
[0112] Step S144: Calculate the first coefficient and the second coefficient of the browser's memory; the first coefficient is the ratio of the amount of memory used to the amount of allocated memory; the second coefficient is the ratio of the amount of memory remaining to the amount of allocated memory.
[0113] Step S146: Based on the weights of the first coefficient and the second coefficient, perform a weighted sum of the first coefficient and the second coefficient to obtain the browser's memory coefficient.
[0114] The weights of the first coefficient and the second coefficient mentioned above are set differently based on the specific application scenario. This embodiment does not make specific limitations here, as long as the sum of the weights of the first coefficient and the second coefficient is 1.
[0115] Steps S142 to S146 above calculate the browser's memory coefficient. Determining the browser's memory coefficient facilitates the subsequent determination of a suggested value for the browser's available memory.
[0116] In one embodiment, the above-mentioned big data rendering method further includes:
[0117] Step S240: When the received data volume of the data to be rendered is less than or equal to the maximum data volume that the browser can render, the data to be rendered is rendered directly.
[0118] Data rendering can be divided into two modes: normal rendering and intelligent rendering. When the amount of data received is less than or equal to the maximum amount of data that the browser can render, the browser can render the data normally, which is the normal rendering mode. When the amount of data received is greater than the maximum amount of data that the browser can render, the browser cannot render the data normally and needs to optimize the data until the browser can render it normally before rendering it. This rendering mode is called intelligent rendering mode.
[0119] This second embodiment provides another big data rendering method for the server side (i.e., the server). Figure 3 This is a flowchart of the big data rendering method in this embodiment, such as... Figure 3 As shown, the process includes the following steps:
[0120] Step S310: In response to the received data analysis request from the user, perform data analysis on the data to be rendered, determine the target optimization information in the data to be rendered, and return the target optimization information to the user; the target optimization information consists of data optimization items that optimize the data volume of the data to be rendered to be less than or equal to the maximum data volume that the browser can render, and description information corresponding to the data optimization items.
[0121] The aforementioned response to a data analysis request received from the user client, performing data analysis on the data to be rendered to determine the target optimization information within the data, can be a large model server on the server side. This large model server includes a pre-set large model. This pre-set large model is a pre-trained model capable of responding to a data analysis request received from the user client, performing data analysis on the data to be rendered, and determining the target optimization information within the data.
[0122] Step S320: In response to the received optimization request for the data to be rendered from the user terminal, optimize the data to be rendered according to the data optimization items in the optimization request to obtain the optimized data to be rendered, and send it to the user terminal for rendering; the optimization request is generated by the user terminal based on the revision information of the user for the target optimization information received from the user.
[0123] In steps S310 to S320 above, the server, in response to a received data analysis request from the user, performs data analysis on the data to be rendered, determines the target optimization information in the data to be rendered, and returns the target optimization information to the user. Then, in response to a received optimization request for the data to be rendered from the user, it optimizes the data to be rendered according to the data optimization items in the optimization request, obtains the optimized data to be rendered, and sends it to the user for rendering. Based on the received data analysis request from the user, the server generates a function to optimize the data volume of the data to be rendered to a level less than or equal to the browser's renderable size. The system obtains the maximum target optimization information for the data to be rendered. Then, it receives optimization requests from the user based on the target optimization information. According to the optimization requests, without restricting the data query conditions, it optimizes the data to be rendered to obtain optimized data with a data volume less than or equal to the maximum data volume that the browser can render. This optimized data is then provided to the user for rendering, ensuring that the browser can render the data normally and avoiding situations such as excessively long loading times, page lag, or even browser crashes. This solves the problem of excessively long loading times, page lag, or even browser crashes in existing big data rendering methods when the data volume exceeds the browser's capacity limit.
[0124] Further, in one embodiment, prior to step S310, the following steps are included:
[0125] Step S301: In response to the data mining instruction received from the user terminal for the data to be rendered, perform data mining operation to obtain the data to be rendered; the data mining instruction is generated by the user terminal based on the data query conditions selected by the user.
[0126] When the web server receives a data mining instruction from the user client regarding the data to be rendered, it calls the data query service. The data query service performs data mining operations based on the data query conditions in the data mining instruction, obtains the data to be rendered, and returns the obtained data to the web server.
[0127] Preferably, after receiving the data to be rendered sent by the data query service, the web server can perform noise reduction processing on the data to be rendered, remove dirty data from the data, obtain the noise-reduced data to be rendered, and send the noise-reduced data to the client, while simultaneously storing the noise-reduced data to be rendered in the server's cache.
[0128] This third embodiment provides another method for rendering large data. Figure 4 This is a flowchart of the big data rendering method in this embodiment, applied to a big data rendering system. The big data rendering system includes a user terminal and a server terminal, such as... Figure 4 As shown, the process includes the following steps:
[0129] Step S410: On the user side, in response to the data query conditions selected by the user, a data mining instruction for the data to be rendered is generated, and the generated data mining instruction is sent to the server.
[0130] In step S420, the server responds to the received data mining instruction by performing a data mining operation to obtain the data to be rendered and then sends the obtained data to the user.
[0131] In this step, the server-side web server responds to the received data mining instructions, performs data mining operations, obtains the data to be rendered, and sends the obtained data to the user's client.
[0132] Step S430: On the user side, the data to be rendered is received, and the data volume of the received data to be rendered is determined; the maximum data volume that the browser can render is determined; it is determined whether the data volume of the received data to be rendered is greater than the maximum data volume that the browser can render; when the data volume of the received data to be rendered is greater than the maximum data volume that the browser can render, a data analysis request is sent to the server.
[0133] The aforementioned client, after receiving the data to be rendered, can also cache the data in the client's cache. Sending the data analysis request to the server can be achieved by the browser sending a data intelligence analysis API request to the web service. The request parameters must be consistent with the data mining instruction interface for the data to be rendered to ensure that the analyzed data is indeed the data to be rendered.
[0134] Step S440: The server responds to the received data analysis request by performing data analysis on the data to be rendered, determining the target optimization information in the data to be rendered, and returning the target optimization information to the user. The target optimization information consists of data optimization items that optimize the data volume of the data to be rendered to be less than or equal to the maximum data volume that the browser can render, and description information corresponding to the data optimization items.
[0135] Step S450: On the user side, the received target optimization information is displayed; and based on the user's revision information of the target optimization information, an optimization request for the data to be rendered is generated and sent to the server.
[0136] In step S460, the server responds to the received optimization request, optimizes the data to be rendered according to the data optimization items in the optimization request, obtains the optimized data to be rendered, and sends it to the user.
[0137] Step S470: On the user side, the received optimized data to be rendered is rendered.
[0138] Steps S410 to S470 above are performed through interaction between the user client and the server. The user client, in response to the data query conditions selected by the user, generates a data mining instruction for the data to be rendered. The server mines the data to be rendered according to the data mining instruction. The user client determines whether the received data volume of the data to be rendered is greater than the maximum data volume that the browser can render. If the received data volume of the data to be rendered is greater than the maximum data volume that the browser can render, the user client sends a data analysis request to the server. The server, in response to the received data analysis request, performs data analysis on the data to be rendered to determine the target optimization information in the data to be rendered. The user client, based on the user's revision information of the target optimization information, generates an optimization request for the data to be rendered. Then, the server, in response to the received data analysis request... Upon receiving an optimization request, the system optimizes the data to be rendered based on the data optimization items in the request, resulting in optimized data to be rendered. Finally, the user client renders the received optimized data. Throughout the process, optimization is only performed when the browser is unable to render the data, until it can be rendered. Without restricting data query conditions, the system optimizes the data volume of the data to be rendered to be less than or equal to the maximum data volume that the browser can render, ensuring that the browser can render the data normally and avoiding excessively long loading times, page lag, or even browser crashes. This solves the problem of excessively long loading times, page lag, or even browser crashes in existing big data rendering methods when the data volume exceeds the browser's capacity limit.
[0139] In another embodiment, the server includes a web server and a large model server. In step S440, the server, in response to a received data analysis request, performs data analysis on the data to be rendered, determines the target optimization information in the data to be rendered, and returns the target optimization information to the user, including:
[0140] Step S442: The web server receives a data analysis request from the user and sends the received data analysis request to the large model server.
[0141] Step S444: The large model server performs data analysis on the data to be rendered based on the received data analysis request, determines the target optimization information in the data to be rendered, and returns the target optimization information to the WEB server.
[0142] Step S446: The web server sends the received target optimization information to the user terminal.
[0143] Steps S442 to S446 above involve the interaction between the web server and the large model server to respond to received data analysis requests, perform data analysis on the data to be rendered, determine the target optimization information in the data to be rendered, and return the target optimization information to the user.
[0144] Step S460: The server, in response to the received optimization request, optimizes the data to be rendered according to the data optimization items in the optimization request, obtains the optimized data to be rendered, and sends it to the user, including:
[0145] Step S462: The WEB server receives an optimization request for the data to be rendered from the user and sends the received optimization request to the large model server; the optimization request is generated by the user based on the revision information received from the user regarding the target optimization information.
[0146] In this step, the aforementioned optimization request involves the user inputting natural language with generalizable characteristics to revise the data optimization items based on the data optimization items and their corresponding descriptions in the target optimization information. This revision information, obtained through natural language editing, facilitates accurate data recognition by the large model server. After receiving the optimization request for the data to be rendered from the user, the web server simultaneously verifies the legality of the natural language corresponding to the data optimization items. Upon successful verification, the received optimization request is sent to the large model server, further ensuring that the optimization request received by the large model server can be accurately recognized, thereby guaranteeing the accuracy of the optimization processing of the data to be rendered.
[0147] In step S464, the large model server optimizes the data to be rendered based on the data optimization items in the received optimization request, obtains the optimized data to be rendered, and sends it to the WEB server.
[0148] In step S466, the web server sends the received optimized data to be rendered to the client for rendering.
[0149] Steps S462 to S466 above involve the interaction between the web server and the large model server to respond to received optimization requests, optimize the data to be rendered according to the data optimization items in the optimization request, obtain the optimized data to be rendered, and send it to the user terminal.
[0150] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0151] Based on the same inventive concept, this embodiment also provides a big data rendering system, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as described above. The terms "module," "unit," "subunit," etc., used below can refer to combinations of software and / or hardware that achieve a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.
[0152] In one embodiment, Figure 5 This is a structural block diagram of a big data rendering system provided in an embodiment of this application, as shown below. Figure 5 As shown, the big data rendering system includes: a user terminal 52 and a server terminal 54; the user terminal 52 is used to execute any one of the big data rendering methods in Embodiment 1 above; the server terminal 54 is used to execute any one of the big data rendering methods in Embodiment 2 above.
[0153] In one embodiment, the user end of the aforementioned big data rendering system can also perform page display and human-computer interaction, which can be intelligent rendering pop-up layout and interaction.
[0154] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.
[0155] In one embodiment, an electronic device is provided, including a memory and a processor. The memory is connected to the processor and is used to store a program. The processor is used to implement any one of the big data rendering methods in Embodiments 1, 2, and 3 above by running the program in the memory.
[0156] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.
[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A big data rendering method, characterized in that, The method includes: When the received data volume of the data to be rendered is greater than the maximum data volume that the browser can render, a data analysis request is sent to the server. The data analysis request is used to instruct the server to perform data analysis on the data to be rendered and determine the target optimization information in the data to be rendered. The target optimization information consists of data optimization items that optimize the data volume of the data to be rendered to be less than or equal to the maximum data volume that the browser can render, and description information corresponding to the data optimization items. The server receives the target optimization information returned by the server and generates an optimization request for the data to be rendered based on the user's revision information regarding the target optimization information. The server then sends the optimization request to the server. The optimization request instructs the server to optimize the data to be rendered according to the data optimization items in the optimization request to obtain the optimized data to be rendered. Receive the optimized data to be rendered sent by the server, and perform data rendering on the received optimized data to be rendered.
2. The big data rendering method according to claim 1, characterized in that, Before sending a data analysis request to the server when the received data volume to be rendered exceeds the maximum data volume that the browser can render, the following steps are included: Receive the data to be rendered sent by the server and determine the data volume of the received data to be rendered; Determine the maximum amount of data that the browser can render; Determine whether the received data volume of the data to be rendered is greater than the maximum data volume that the browser can render.
3. The big data rendering method according to claim 2, characterized in that, Determining the maximum amount of data that the browser can render includes: Determine the available memory information of the browser; Based on the current scene and a preset scene memory mapping table, the memory value occupied by unit data rendering in the current scene is determined; the preset scene memory mapping table is a mapping table between each scene and the memory value occupied by unit data rendering corresponding to each scene. Based on the browser's available memory information and the memory value occupied by rendering a unit of data in the current scene, the maximum amount of data that the browser can render is determined.
4. The big data rendering method according to claim 3, characterized in that, Determining the browser's available memory information includes: Based on the baseline value of the browser's available memory and the browser's memory coefficient, a recommended value for the browser's available memory is determined; The maximum safe value of available memory for the browser was determined using a security isolation test. The smaller of the suggested value of the browser's available memory and the maximum safe value of the browser's available memory is determined as the browser's available memory information.
5. The big data rendering method according to claim 4, characterized in that, The process of determining a suggested value for the browser's available memory based on a baseline value of the browser's available memory and the browser's memory coefficient includes: A baseline value for the browser's available memory is determined using either a first calculation method or a second calculation method; the first calculation method is a standard API calculation method; the second calculation method is a calculation method combining the number of CPU cores and UA detection. Based on the monitoring results of the browser's JS heap usage, the browser's memory coefficient is determined; The recommended value for the browser's available memory is determined by multiplying the baseline value of the browser's available memory by the browser's memory coefficient.
6. The big data rendering method according to claim 5, characterized in that, The determination of the browser's memory coefficient based on the monitoring results of the browser's JS heap usage includes: Based on the monitoring results of the browser's JS heap usage, data information on the browser's JS heap usage is determined; the data information includes the amount of allocated memory, the amount of memory used, and the amount of memory remaining. Calculate a first coefficient and a second coefficient for the browser's memory usage; the first coefficient is the ratio of the used memory to the allocated memory; the second coefficient is the ratio of the remaining memory to the allocated memory. Based on the weights of the first coefficient and the second coefficient, the first coefficient and the second coefficient are weighted and summed to obtain the browser's memory coefficient.
7. A big data rendering method, characterized in that, The method includes: In response to a received data analysis request from the user, the system performs data analysis on the data to be rendered, determines the target optimization information in the data to be rendered, and returns the target optimization information to the user. The target optimization information consists of data optimization items that optimize the data volume of the data to be rendered to a level less than or equal to the maximum data volume that the browser can render, and description information corresponding to the data optimization items. In response to an optimization request received from the user terminal for the data to be rendered, the data to be rendered is optimized according to the data optimization items in the optimization request to obtain optimized data to be rendered, and then sent to the user terminal for rendering; the optimization request is generated by the user terminal based on the revision information received from the user regarding the target optimization information.
8. A big data rendering method, characterized in that, Applied to a big data rendering system, the big data rendering system including a user terminal and a server terminal, the method includes: The user terminal, in response to the data query conditions selected by the user, generates a data mining instruction for the data to be rendered and sends the generated data mining instruction to the server. The server, in response to the received data mining instruction, performs data mining operations to obtain the data to be rendered, and sends the obtained data to be rendered to the user terminal; The user terminal receives the data to be rendered, determines the data volume of the received data to be rendered, determines the maximum data volume that the browser can render, determines whether the data volume of the received data to be rendered is greater than the maximum data volume that the browser can render, and when the data volume of the received data to be rendered is greater than the maximum data volume that the browser can render, sends a data analysis request to the server. The server, in response to the received data analysis request, performs data analysis on the data to be rendered, determines the target optimization information in the data to be rendered, and returns the target optimization information to the user terminal; the target optimization information consists of data optimization items that optimize the data volume of the data to be rendered to be less than or equal to the maximum data volume that the browser can render, and description information corresponding to the data optimization items. The user terminal displays the received target optimization information; and based on the user's revision information of the target optimization information, generates an optimization request for the data to be rendered, and sends the optimization request to the server. The server, in response to the received optimization request, optimizes the data to be rendered according to the data optimization items in the optimization request, obtains the optimized data to be rendered, and sends it to the user terminal. The user terminal performs data rendering on the received optimized data to be rendered.
9. The big data rendering method according to claim 8, characterized in that, The server includes a web server and a large model server, and the method includes: The web server receives data analysis requests from the user and sends the received data analysis requests to the large model server. The large model server, based on the received data analysis request, performs data analysis on the data to be rendered, determines the target optimization information in the data to be rendered, and returns the target optimization information to the WEB server; The web server sends the received target optimization information to the user terminal; The web server receives an optimization request for the data to be rendered from the user client and sends the received optimization request to the large model server; the optimization request is generated by the user client based on the revision information received from the user regarding the target optimization information; The large model server optimizes the data to be rendered based on the data optimization items in the received optimization request, obtains the optimized data to be rendered, and sends it to the WEB server. The web server sends the received optimized data to be rendered to the user terminal for rendering.
10. A big data rendering system, characterized in that, The system includes: a user terminal and a server terminal; The user terminal is used to execute the big data rendering method according to any one of claims 1 to 6; The server is used to execute the big data rendering method as described in claim 7.