A method for uploading and displaying large data sets for AI training

Abstract

The present application relates to a method for uploading and displaying a large data set for AI training, including a client dividing a large data set file into multiple small file blocks and caching them in IndexedDB, reading a target file block from IndexedDB and uploading it to a server, deleting the target file block in IndexedDB when the upload is completed, until there is no file block in IndexedDB, the client sending a signal to the server indicating that the file upload is complete, the client creating a cross observer and a signal object, the client loading a page and sending a request to the server, parsing a visual chart preview returned by the server, adjusting the visual chart preview to obtain a placeholder chart, using the placeholder chart to occupy the actual chart position, the client parsing the training result data, triggering a visual chart drawing operation in the visual window of the front-end chart component, updating the view, and replacing the placeholder chart.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for uploading and displaying a large data set for AI training. Background Art

[0002] With the rapid development of artificial intelligence technology, large data sets play an increasingly important role in AI training. However, the uploading and displaying of large data sets still face some challenges: first, the traditional uploading method requires the entire file to be read into memory. When the browser is closed or the upload is interrupted, the data will be lost and the user has to reselect the file, which reduces the success rate of the upload; second, it takes a long time for the client to process large data sets, and a large amount of blank space may appear on the front-end interface during this period, affecting the user's immediate perception of the data; finally, when displaying data, all the data in the page is loaded at once, resulting in a long initial loading time, causing unnecessary data interaction and wasting network resources. Therefore, there is an urgent need for an efficient large data set upload and display solution to improve the upload speed and success rate, reduce the front-end blank time, and optimize the data display performance.

[0003] Compared with existing technologies, existing big data uploading and display technologies have the following main problems:

[0004] 1. Upload lacks a more effective resume mechanism. When abnormal situations such as power outages occur, you need to reselect the file to upload.

[0005] 2. When the client processes a large data set, it takes a long time and a large amount of blank space may appear on the front-end interface, affecting the user experience.

[0006] 3. All data is loaded at once when displaying data, which takes a long time for initial loading, wastes network resources, and has a slow loading speed.

[0007] 4. The client renders a large data set display page at one time, loading a large amount of data each time, which places high demands on client performance and affects the page loading speed.

[0008] 5. Lack of decoupling mechanism for user operations and data leads to poor system flexibility. Summary of the invention

[0009] To solve the above problems, the present invention provides a method for uploading and displaying a large AI training data set, which adopts the following technical solutions:

[0010] A method for uploading and displaying a large AI training data set includes: an uploading step; the uploading step includes:

[0011] Step S101, the client divides the large data set file selected by the user to upload into multiple small file blocks, and caches the multiple small file blocks into the local database IndexedDB of the browser;

[0012] Step S102, the client reads the target file block and uploads the target file block to the server, where the target file block is one of the multiple file blocks stored in the local database IndexedDB of the browser;

[0013] Step S103, when the client finishes uploading the target file block, the target file block is deleted from the local database IndexedDB of the browser;

[0014] Step S104, the client determines whether there is a small file block in the local database IndexedDB of the browser, if yes, then continue to execute step S102, if not, then continue to execute step S105;

[0015] Step S105, the client sends a signal to the server indicating that the file upload is complete;

[0016] In another possible implementation manner, a display step is further included, and the display step includes:

[0017] Step S201, the client creates an intersection observer IntersectionObserver and creates a signal object signals;

[0018] Step S202, the client loads the page and sends a request to the server. After receiving the request, the server processes the request and then feeds back the code including the chart layout rendered by the server, and injects it into the client rendering page. The request sent by the client to the server carries basic configuration information including the chart type.

[0019] Step S203, the client parses the preview image of the visualization chart after training the large data set returned by the server in response to the request sent by the client, and adjusts the preview image of the visualization chart to obtain a placeholder chart, and uses the placeholder chart to occupy the actual chart position;

[0020] Step S204, the client requests the server to process the training result data of the large data set according to the visual chart in the visual window and parses it, triggers the drawing operation of the visual chart in the visual window of the front-end chart component, and updates the view to replace the placeholder chart occupied in step S203. The visual window is a pre-set area on the client page that can display the large data set, and the visual chart is a chart with a display ratio reaching a preset ratio in the visual window;

[0021] Step S205, after the client monitors the response of the intersection observer to the overlap of all the charts in the page with the visible window, if a new chart appears in the visible window, execute step S207;

[0022] Step S206, the client injects the result of the change of the time range or refresh frequency in the filter control into the signal object signals. When the client chart component monitors the update of the signal object signals, it jumps to step S207;

[0023] Step S207, the client requests the server to process the training result data of the large data set and parses it according to the visual chart in the visual window, triggering the visual chart drawing operation in the visual window of the front-end chart component. If there is a placeholder chart at the actual chart position, the placeholder chart is replaced, otherwise the chart is updated.

[0024] In another possible implementation, the large data set file includes multiple files, the client divides the large data set file selected by the user to upload into multiple small file blocks, and caches the multiple small file blocks in the local database IndexedDB of the browser, including:

[0025] The client divides the large data set file selected by the user to upload into multiple file blocks, generates corresponding hash identifiers for the multiple file blocks, and generates corresponding file IDs for the multiple files;

[0026] The multiple file blocks are uniformly stored in the browser's local database IndexedDB according to the Blob type, combined with the file IDs corresponding to the multiple files and the hash identifiers corresponding to the multiple file blocks.

[0027] In another possible implementation, the file chunks in the local database IndexedDB have corresponding status flags, including: to be uploaded and being uploaded. The target file chunk is the first file chunk among the file chunks to be uploaded whose status flag is found in the local database IndexedDB according to the file ID.

[0028] In another possible implementation, in step S202 , the code of the chart layout rendered by the server is injected into the client rendering page by using the client activation technology.

[0029] In another possible implementation, the client parses a preview of a visualization chart trained on a large data set returned by the server in response to a request sent by the client, and adjusts the preview of the visualization chart to obtain a placeholder chart, including:

[0030] The server generates a preview of the visualization chart using backend rendering technology according to the basic configuration information in the request sent by the client, and returns the preview of the visualization chart to the client;

[0031] The client adjusts the preview image of the visualization chart based on the actual chart position and the size of the actual chart corresponding to the actual chart position to obtain a placeholder chart.

[0032] In summary, this application includes the following beneficial technical effects:

[0033] This application adopts a block upload strategy, which divides a large data set into multiple small blocks, and each small block is uploaded to the distributed storage system independently, thereby improving the upload speed and success rate; and, before uploading the large data set, the client caches the file in the browser's local database (IndexedDB) to ensure that the file can be automatically continued to be uploaded after an accident causes the upload to be interrupted; in addition, this application also implements a multi-dimensional and efficient display technology, which reduces the client's loading time through client activation technology (a method of injecting server-rendered content into the client-rendered page after the page is loaded), reduces waiting time by using low-resolution images to take up space, and reduces unnecessary data transmission through defer loading technology (DeferLoading), etc., to display data faster and more efficiently; at the same time, by using the signal object (signals) mechanism on the client to conditionally update the chart, the decoupling between user operations and data updates is achieved, thereby improving modularity and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flowchart of a method for uploading and displaying a large AI training data set in an embodiment of the present application.

[0035] Figure 2 It is a flow chart of a method for rendering a page on a client side in an embodiment of the present application.

[0036] Figure 3 This is an example diagram of the relationship between the chart and the browser page in the embodiment of the present application.

[0037] Figure 4 It is a flowchart of the deferred loading technology in the embodiment of the present application. DETAILED DESCRIPTION

[0038] The present application is further described in detail below in conjunction with the accompanying drawings.

[0039] After reading this specification, those skilled in the art may make non-creative modifications to this embodiment as needed, but such modifications are protected by patent law as long as they are within the scope of the claims of this application.

[0040] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] In addition, the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article, unless otherwise specified, generally means that the associated objects before and after are in an "or" relationship.

[0042] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0043] like Figure 1 As shown, a method for uploading and displaying a large AI training data set includes:

[0044] Step S101 : the client divides the large data set file selected for uploading by the user into a plurality of small file blocks, and caches the plurality of small file blocks into the local database IndexedDB of the browser.

[0045] For the embodiment of the present application, IndexedDB is the local database of the browser, which is a non-relational database that allows the storage of large amounts of data, supports the storage of Blob (Binary Large Object, binary type large object) type data, and provides a search interface. Since its operation is asynchronous, the storage and search of small file blocks will not block other user operations. Step S101 is an initialization operation for the large data set file to be uploaded. In the embodiment of the present application, at the client, the AI ​​large data set file selected by the user to be uploaded is usually a large file. The client divides the large file into multiple small file blocks. Specifically, the division standard is to determine the size of the divided file blocks (in the embodiment of the present application, the size of the file blocks is 100MB) according to the hash processing time of the client for the file and the number of server APIs that need to be called when uploading the file, so that the divided file blocks are suitable for network transmission.

[0046] Furthermore, in an embodiment of the present application, the large data set file includes multiple files, and the client will generate a unique file ID for each file to facilitate subsequent tracking and identification of each file block. For the multiple divided file blocks, the client will generate a corresponding hash identifier for each file block to prepare for file integrity verification during subsequent upload. Since IndexedDB supports the storage of Blob type data, the original data of the multiple file blocks are stored in the browser's local database IndexedDB according to the Blob type, combined with the file IDs corresponding to the multiple files and the hash identifiers corresponding to the multiple file blocks. This enables the local database IndexedDB to conveniently store the file blocks of the large data set file to complete the initial operation before uploading the large data set file.

[0047] When an abnormal situation occurs during the upload process and the upload is interrupted, the client reloads the page to determine whether there are cached file blocks in IndexedDB. If so, it continues to read file blocks from IndexedDB and uploads the read file blocks. After the upload is completed, the corresponding file blocks are deleted from IndexedDB until there are no cached file blocks in IndexedDB. When there are no cached file blocks in IndexedDB, it means that all large data set files have been uploaded successfully when the interruption occurred.

[0048] By storing in blocks, the memory and network bandwidth are avoided when uploading large data sets; by generating a unique file ID for each file, it is convenient to quickly identify and track it during subsequent uploads; by using the asynchronous operation characteristics of IndexedDB, it is avoided that the user's other operations are blocked when storing and searching files. When an abnormal situation occurs during the upload process, after the client reloads the page, if there is a cached file block in IndexedDB, the cached file block is read and the API is called to continue the upload operation, realizing the interrupted upload function.

[0049] In step S102, the client reads the target file block and uploads the target file block to the server. The target file block is one of the multiple file blocks stored in the local database IndexedDB of the browser.

[0050] Step S103, when the client finishes uploading the target file block, the target file block is deleted from the local database IndexedDB of the browser.

[0051] Step S104, the client determines whether there is a small file block in the browser's local database IndexedDB, if so, continue to execute step S102, if not, continue to execute step S105.

[0052] Step S105: the client sends a signal to the server indicating that the file upload is complete.

[0053] For the embodiment of the present application, in the local database IndexedDB, the file block has a corresponding status flag, and the status flag includes: to be uploaded and being uploaded. The target file block is the first file block in the file blocks to be uploaded whose status flag is found according to the file ID in the local database IndexedDB. The client determines the target file block from multiple file blocks, sends the target file block and the corresponding hash identifier to the server through the file upload API provided by the server, and changes the status flag of the target file block to being uploaded. When the file upload API of the server responds successfully, the target file block is deleted from the local database IndexedDB. The client determines whether there is a corresponding file block in the local database IndexedDB of the current browser through the file ID. When the file block included in the file ID does not exist, it means that the large data set file selected by the current user to upload has been uploaded. The client sends a signal to the server that the file upload is completed. When the server receives the signal sent by the client that the file upload is completed, the server verifies the file according to the hash identifier corresponding to each file block, and integrates the file blocks in time, thereby completing the upload operation of the entire large data set file.

[0054] By deleting each small file block from the local database IndexedDB in time after uploading, the space occupied by the local database IndexedDB is reduced, the space utilization rate of the local database IndexedDB is improved, and the search performance of the local database IndexedDB is optimized.

[0055] Step S201: The client creates an intersection observer IntersectionObserver and creates a signal object signals.

[0056] Step S202, the client loads the page and sends a request to the server. After receiving the request, the server processes the request and feeds back the code including the chart layout rendered by the server, and injects it into the client rendering page. The request sent by the client to the server carries basic configuration information including the chart type.

[0057] For the embodiment of the present application, the client activation technology is a front-end optimization technology. Its working principle is: after the page is loaded, the client sends a request carrying basic configuration information to the server, the server receives the basic configuration information, and uses the back-end rendering technology to generate HTML (HyperTextMarkup Language, Hypertext Markup Language), CSS (Cascading Style Sheets, Cascading Style Sheets) and JavaScript (an advanced, interpreted programming language) code that describes the layout, style and interaction logic of the front-end components. These codes are returned to the client as static resources. After the client receives these static resources, it dynamically combines these static resources returned by the server with the client's local HTML, CSS and JavaScript resources to achieve complete rendering of the page and maintain state synchronization with the server.

[0058] Specifically, if Figure 2 As shown, after the page is loaded, the client sends a request to the server, and the request carries the basic configuration information of the chart, including the chart type, data source, etc. After receiving the request, the server uses the back-end rendering technology to generate HTML, CSS and JavaScript codes that describe the layout, style and interaction logic of the chart according to the basic configuration information sent by the front-end. These codes are returned to the client as static resources. After receiving these static resources, the client searches for components corresponding to these static resources. In the components found, the client combines the local HTML, CSS and JavaScript resources with the static resources returned by the server and completes the rendering, maintaining state synchronization with the server. In the subsequent front-end interaction process, the client uses these components to process user interaction operations and integrate dynamic content into the existing DOM structure without reloading the entire page. Compared with the existing technology, the client activation technology can make full use of the server computing power, reduce the client rendering pressure, avoid the performance bottleneck caused by the client rendering, and significantly reduce the initial loading time of the front end, improve the response speed of the page, improve the overall performance, and maintain the consistency of the front and back ends, avoiding the interaction problems caused by the inconsistency of the front and back ends.

[0059] In step S203, the client parses the preview image of the visualization chart after training the large data set returned by the server in response to the request sent by the client, and adjusts the preview image of the visualization chart to obtain a placeholder chart, and uses the placeholder chart to occupy the actual chart position.

[0060] In step S204, the client requests the server to process the training result data of the large data set and parse it according to the visual chart in the visual window, triggering the drawing operation of the visual chart in the visual window of the front-end chart component, and updating the view to replace the placeholder chart in step S203. The visual window is a pre-set area on the client page that can display the large data set, and the visual chart is a chart with a display ratio reaching a preset ratio in the visual window.

[0061] For the embodiment of the present application, the image placeholder technology is a method in which the client enlarges a low-resolution chart rendered by the server that is smaller in size but consistent with the actual content to a size consistent with the actual chart and inserts it into the DOM position of the actual chart before the actual high-definition chart is drawn.

[0062] Specifically, after the page is loaded, the client sends a request to the server, which carries the basic configuration information of the chart, including the chart type and data source. Based on the basic configuration information sent by the front-end, the server uses the back-end rendering technology to generate a small-size, low-resolution chart in SVG (Scalable Vector Graphics) format, and returns it to the client as a visual chart preview. After receiving the visual chart preview, the client makes appropriate adjustments to the visual chart preview according to the size and position of the actual chart to obtain a placeholder chart, and then inserts the placeholder chart into the DOM location where the actual chart is located (such as Figure 3 (b)). At the same time, the client listens to the more complete training result data returned by the server, and parses the training result data to generate the data format required by the chart component to draw the chart. The client chart component draws the visual chart according to the training result data with the changed data format, and replaces the previous placeholder chart with the drawn high-definition chart. The effect after replacement is as follows Figure 3 (c) Figure 3 In (c), the two charts shown at the top of the visible window have a display ratio greater than 30% in the visible window, so the client actually draws the two charts and replaces the previous placeholder charts with the drawn high-definition icons. The chart shown at the lower left corner of the visible window has a display ratio less than 30% in the visible window and is an invisible chart. The client does not draw it, and only displays a small-size, low-resolution placeholder chart for this chart.

[0063] Furthermore, in the embodiment of the present application, the chart component uses a progressively enhanced transition effect to make the entire process smoother. The placeholder chart returned by the server reduces the client request response time due to its small size and low resolution; at the same time, due to the characteristics of SVG, the client can still maintain its high clarity after enlarging the placeholder chart. This technology ensures visual effects due to faster loading speed and effects similar to actual charts, reducing user waiting time.

[0064] Step S205, after the client monitors the response of the intersection observer to the overlap of all the charts in the page with the visible window, if a new chart appears in the visible window, step S207 is executed.

[0065] For the embodiment of the present application, the client also uses the deferred loading technology when drawing the chart. The deferred loading (DeferLoading) technology is a method for delaying the loading of non-urgent resources without affecting the initial loading speed of the page. This embodiment uses the intersection observer (Intersection Observer) to calculate the chart in the visible window. The intersection observer (Intersection Observer) is an API provided by the browser for asynchronously detecting whether the target element and its ancestor or top-level element intersect. It can load the element when it enters the visible window and unload it when it leaves by monitoring the intersection state changes between the target element and its ancestor element, thereby achieving the effect of deferred loading and improving the page loading speed. When the target element intersects with its ancestor element, a callback function will be triggered. In the callback function, the resources of the target element can be decided to be loaded or unloaded based on the intersection observation results.

[0066] Specifically, this embodiment defines the area within 20 pixels from the upper, lower, left, and right boundaries of the browser page as the visible window (eg, Figure 3 (a)). When the portion of a single chart that appears in the visible window is greater than or equal to 30% of the chart itself, the client will mark the chart as "visible", that is, the chart is a visible chart (such as Figure 3 (c)), the client writes the above definition into the cross observer. Figure 4 As shown, after the page is loaded, the client annotates the visual chart in the visual window according to the definition of the visual chart (corresponding to Figure 4 In the middle: the chart in the visible window is marked as "visible"). At the same time, the client creates a cross observer for all chart elements (including those outside the visible window) and monitors the proportion of each chart appearing in the visible window. When the user scrolls the page or adjusts the size of the browser window (hereinafter referred to as user adjustment), the proportion of the chart appearing in the visible window changes (such as Figure 3(d) shows that the user scrolls the page, resulting in a new chart appearing in the lower left corner of the visible window). The client triggers the event callback of the cross observer. In the event callback, a "visible" label is added to the chart that appears in the visible window and has a proportion of 30%, and the new chart is determined to be a visible chart; the "visible" label is cancelled for the chart that accounts for less than 30% of the visible window after the user's adjustment (that is, the "visible" label is cancelled for some charts that do not meet the proportion mentioned in the above "labeling the chart in the visible window according to the definition of the visible chart"), so as to avoid loading training result data that is not visible to the user, resulting in the occupation of network resources and consumption of client performance.

[0067] Finally, the client filters the charts with the "visual" annotation and requests the latest training result data of these visual charts from the server, and passes the returned latest training result data to the front-end chart component for drawing, thereby updating the view: Specifically, if the client determines that the chart has a "visual" annotation, it requests the latest training result data of the chart from the server. The client parses the returned latest training result data into the format required by the chart component, and passes it to the front-end chart component for drawing, thereby updating the chart, or replacing the placeholder chart.

[0068] The intersection observer IntersectionObserverAPI runs in the background thread of the browser, does not block the main thread, and improves the responsiveness of the page; each chart element can create an independent intersection observer, which has good flexibility. In this embodiment, the display of the client chart component in steps S204 and S207 involves a large number of visual charts. Especially when the user sets the refresh rate for the filter control, the implementation of this embodiment can greatly reduce the number of frequent requests to the server API, reduce the pressure of the drawing operation of the front-end chart component, significantly improve the loading speed of the page, and improve the user experience.

[0069] In step S206, the client injects the result of the change of the time range or refresh frequency in the filter control into the signal object signals. When the client chart component monitors the update of the signal object signals, it jumps to step S207.

[0070] For the embodiment of the present application, the filter control is an important supplement to the front-end chart component. It provides an intuitive tool set that enables users to view data in a customized manner according to their needs. The filter control contains a time range selector and a refresh frequency selector so that users can quickly locate the time period and data refresh speed they want to know. In the implementation of the front-end chart component, we created a signal object signals, which serves as an interactive bridge between users and charts. Through the filter control, users can easily select different time ranges (such as "nearly one week", "nearly 10 days") and refresh frequencies (such as "every 1 minute", "every 5 minutes"). Each user's selection will trigger the generation of signals, which will carry information about the time range and refresh frequency, and will be managed and tracked by the signal object.

[0071] Specifically, in this embodiment, the front-end chart component continuously monitors the signal object signals. Once a signal is received, it will send a request to the server API according to the time range and refresh frequency contained in the signal to obtain the latest large data set training result data. The front-end chart component will process and display the training result data to reflect the most intuitive visual chart to the user. In addition, when the user selects a time range with a fixed start and end time, the system will automatically adjust the refresh frequency to no refresh. In this mode, the signals signal will only contain time range information. The chart component will monitor these signals and obtain the historical training result data of the corresponding time range from the server API based on the received signal content, and then update the chart to reflect the historical data trend. In this way, the filtering control not only improves the interactivity of the user interface, but also makes the acquisition and display of data more flexible and efficient, thereby providing users with a richer and customized data experience.

[0072] Step S207, the client requests the server to process the training result data of the large data set and parses it according to the visual chart in the visual window, triggering the visual chart drawing operation in the visual window of the front-end chart component. If there is a placeholder chart at the actual chart position, the placeholder chart is replaced, otherwise the chart is updated.

[0073] For the embodiment of the present application, after step S204, since the browser page has displayed at least one visible chart at this time, when the user has operations such as scrolling the page or zooming the page or filtering the required data, it is possible that after the user performs the operation, the visible window needs to redisplay the position where the drawn chart has been drawn for other charts, that is, the other previously drawn charts have replaced the placeholder chart. At this time, the chart at the corresponding position is updated, and the previously drawn chart is replaced by the redisplayed drawn chart; however, there is another situation at the same time: if Figure 3As shown in (c), since the display ratio of the chart in the lower left corner in the visible window is less than 30%, for this chart, there is only a placeholder chart in its relative position in the visible window, and it has not been actually drawn. Therefore, when other charts need to be displayed at this position after adjustment by the user, that is, when other charts need to be drawn at this position, and there is a placeholder chart at this position, it is necessary to replace the placeholder chart with the drawn chart.

[0074] In addition, the present embodiment also provides an upload and display device for AI training large data sets, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the upload and display method of AI training large data sets. In addition, the present embodiment also provides a computer-readable storage medium, in which a computer program / instruction is stored, and the computer program / instruction is programmed or configured to execute the upload and display method of AI training large data sets through a processor. In addition, the present embodiment also provides a computer program product, including a computer program / instruction, which is programmed or configured to execute the upload and display method of AI training large data sets through a processor.

[0075] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0076] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for uploading and displaying a large AI training data set, characterized in that: include: Upload steps: The uploading step includes: Step S101, the client divides the large data set file selected by the user to upload into multiple small file blocks, and caches the multiple small file blocks into the local database IndexedDB of the browser; Step S102, the client reads the target file block and uploads the target file block to the server, where the target file block is one of the multiple file blocks stored in the local database IndexedDB of the browser; Step S103, when the client finishes uploading the target file block, the target file block is deleted from the local database IndexedDB of the browser; Step S104, the client determines whether there is a small file block in the local database IndexedDB of the browser, if yes, then continue to execute step S102, if not, then continue to execute step S105; Step S105, the client sends a signal to the server indicating that the file upload is complete; The present invention also includes: a display step; the display step includes: Step S201, the client creates an intersection observer IntersectionObserver and creates a signal object signals; Step S202, the client loads the page and sends a request to the server. After receiving the request, the server processes the request and then feeds back the code including the chart layout rendered by the server, and injects it into the client rendering page. The request sent by the client to the server carries basic configuration information including the chart type. Step S203, the client parses the preview image of the visualization chart after training the large data set returned by the server in response to the request sent by the client, and adjusts the preview image of the visualization chart to obtain a placeholder chart, and uses the placeholder chart to occupy the actual chart position; Step S204, the client requests the server to process the training result data of the large data set according to the visual chart in the visual window and parses it, triggers the drawing operation of the visual chart in the visual window of the front-end chart component, and updates the view to replace the placeholder chart occupied in step S203. The visual window is a pre-set area on the client page that can display the large data set, and the visual chart is a chart with a display ratio reaching a preset ratio in the visual window; Step S205, after the client monitors the response of the intersection observer to the overlap of all the charts in the page with the visible window, if a new chart appears in the visible window, execute step S207; Step S206, the client injects the result of the change of the time range or refresh frequency in the filter control into the signal object signals. When the client chart component monitors the update of the signal object signals, it jumps to step S207; Step S207, the client requests the server to process the training result data of the large data set and parses it according to the visual chart in the visual window, triggering the visual chart drawing operation in the visual window of the front-end chart component. If there is a placeholder chart at the actual chart position, the placeholder chart is replaced, otherwise the chart is updated.

2. The method for uploading and displaying a large AI training data set according to claim 1, characterized in that: The large data set file includes multiple files. The client divides the large data set file selected by the user to upload into multiple small file blocks, and caches the multiple small file blocks in the local database IndexedDB of the browser, including: The client divides the large data set file selected by the user to upload into multiple file blocks, generates corresponding hash identifiers for the multiple file blocks, and generates corresponding file IDs for the multiple files; The multiple file blocks are uniformly stored in the browser's local database IndexedDB according to the Blob type, combined with the file IDs corresponding to the multiple files and the hash identifiers corresponding to the multiple file blocks.

3. The method for uploading and displaying a large AI training data set according to claim 1, characterized in that: In the local database IndexedDB, the file chunks have corresponding status flags, including: to be uploaded and being uploaded. The target file chunk is the first file chunk among the file chunks to be uploaded that is found in the local database IndexedDB according to the file ID.

4. The method for uploading and displaying a large AI training data set according to claim 1, characterized in that: In step S202, the code including the chart layout rendered by the server is injected into the client rendering page by using the client activation technology.

5. The method for uploading and displaying a large AI training data set according to claim 1, characterized in that: The client parses the preview image of the visualization chart after training the large data set returned by the server in response to the request sent by the client, and adjusts the preview image of the visualization chart to obtain a placeholder chart, including: The server generates a preview of the visualization chart using backend rendering technology according to the basic configuration information in the request sent by the client, and returns the preview of the visualization chart to the client; The client adjusts the preview image of the visualization chart based on the actual chart position and the size of the actual chart corresponding to the actual chart position to obtain a placeholder chart.

Images