Web-based interactive data visualization creation method

Abstract

The present invention relates to a method for creating interactive data visualization based on web pages, comprising the following steps: obtaining a web page data set based on user search information, obtaining a first-level web page identifier, and sequentially placing the web page identifiers of all sub-web pages of the first-level web page identifier into a first task queue; constructing multiple groups of servers to match the web page identifiers; downloading web page data, and sequentially placing the web page identifiers and downloaded web page data into a second task queue; discarding web page data involving junk web pages; and sequentially rendering the web page data in the second task queue to generate and display a final web page. The present invention improves the download efficiency of web page data from multiple dimensions and reduces resource consumption during web page rendering. At the same time, it can combine a user feedback mechanism to continuously optimize the accuracy of model screening, thereby maximizing the user experience.

Description

Technical Field

[0001] The present invention relates to the technical field of web page data visualization, and in particular to a method for creating interactive data visualization based on a web page. Background Art

[0002] The massive amount of data on the Internet has brought many conveniences to people's lives, but it has also brought huge challenges to server maintenance and operation. How to efficiently and accurately filter out the information users need from massive amounts of data is the focus that browsers need to pay attention to.

[0003] Existing search engine optimization technologies offer a variety of web page data acquisition methods, most of which focus on the data acquisition process itself, improving data acquisition efficiency through the data download process. However, few consider the entire system architecture and improve the overall data communication mechanism. Furthermore, there is no technology that directly identifies spam websites before web page rendering. Summary of the Invention

[0004] To address the shortcomings of the existing technology, the present invention provides a web-based interactive data visualization creation method, comprising the following steps:

[0005] Step S1: Obtain a web page dataset based on user search information;

[0006] Step S2: Based on the web page dataset, obtain a first-level web page identifier;

[0007] Step S3: traverse the webpage IDs of all sub-webpages under the first-level webpage ID, and put the webpage IDs of all sub-webpages into the first task queue in sequence;

[0008] Step S4: construct multiple groups of servers, and match the web page identifiers in the first task queue with the multiple groups of servers;

[0009] Step S5: Each server downloads the web page data based on the matched web page identifier, and sequentially places the web page identifier and the downloaded web page data into the second task queue;

[0010] Step S6: Identify the web page identifiers and web page data in the second task queue, and discard web page data related to junk web pages;

[0011] Step S7: Render the web page data in the second task queue in sequence, generate a final web page, and display it.

[0012] In step S4, each of the multiple server groups constructed includes a proxy server and a download server. The proxy server caches certain web page data, and the download server is used to connect to a specific web page server to download required web page data from the web page server.

[0013] When matching the web page identifiers in the first task queue with multiple groups of servers, a web page identifier is taken out from the first task queue in sequence as the current web page identifier, and the proxy servers in the multiple groups of servers are traversed: if a proxy server has web page data corresponding to the current web page identifier cached, the current web page identifier is matched to the server to which the proxy server belongs; if all proxy servers do not have web page data corresponding to the current web page identifier cached, the current web page identifier is matched to the corresponding server according to the matching order of the multiple groups of servers.

[0014] Wherein, in step S5, when the server downloads the web page data based on the matched web page identifier:

[0015] If the proxy server has web page data corresponding to the web page identifier cached in the proxy server, the web page identifier and the web page data are directly placed in the second task queue in sequence;

[0016] If the proxy server does not cache the web page data corresponding to the web page identifier, the proxy server sends the web page identifier to the download server, and the download server downloads the web page data from the corresponding web page server based on the web page identifier.

[0017] In step S6, web page data related to spam web pages are identified through the following steps:

[0018] Step S61: representing features based on web page identifiers as a first URL matrix, and representing features based on web page data as a first data matrix;

[0019] Step S62: Projecting the first data matrix using the optimal first data projection matrix to obtain a second data matrix, and projecting the first data matrix using the optimal first data projection matrix to obtain a second data matrix;

[0020] Step S63: merging the second data matrix and the second URL matrix to obtain a first input matrix;

[0021] Step S64: inputting the first input matrix into the trained neural network model, and determining whether the web page data involves junk web pages based on the output result of the neural network model.

[0022] In step S62, the method for obtaining the optimal first data projection matrix and the optimal first URL projection matrix is ​​as follows: the first data matrix and the first URL matrix are subjected to multiple typical correlation analysis methods to obtain multiple sets of data projection matrices and URL projection matrices, and the data projection matrix and URL projection matrix with the greatest correlation are selected as the optimal first data projection matrix and the optimal first URL projection matrix.

[0023] In step S64, the training method of the neural network model includes:

[0024] Step S641: preparing a training set, the training set including multiple sets of web page identifiers and web page data corresponding to the web page identifiers, and each set of web page identifiers and web page data is marked, and the marking result is used to indicate whether the web page data involves spam;

[0025] Step S642: for each set of web page identifiers and web page data in the training set, representing features based on the web page identifiers as a third URL matrix, and representing features based on the web page data as a third data matrix;

[0026] Step S643: Projecting the third data matrix using the optimal second data projection matrix to obtain a fourth data matrix, and projecting the third data matrix using the optimal second data projection matrix to obtain a fourth data matrix;

[0027] Step S644: merging the fourth data matrix and the fourth URL matrix to obtain a second input matrix;

[0028] Step S645: For each set of web page identifiers and web page data in the training set, the labeling result obtained based on step S641 and the second input matrix obtained in step S644 are input into the neural network model to train the neural network model.

[0029] The method further includes step S8: constructing a visual annotation menu in the generated final web page, allowing the user to annotate the web page on the visual annotation menu, and adjusting the neural network model based on the annotation results.

[0030] Wherein, in the step S8, a visual annotation menu is constructed by adding a cascading style sheet and a javascript script to the source code of the web page.

[0031] The web-based interactive data visualization creation method provided by the present invention is a systematic mechanism for web URL crawling, web data downloading, junk web page identification, web page generation and feedback. Through various technical means, it improves the download efficiency of web data from multiple dimensions and reduces resource consumption in the web page rendering process. At the same time, it can combine with the user feedback mechanism to continuously optimize the model screening accuracy, thereby maximizing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a logical architecture diagram of the web-based interactive data visualization creation method of the present invention. DETAILED DESCRIPTION

[0033] In order to have a further understanding of the technical solution and beneficial effects of the present invention, the technical solution and beneficial effects of the present invention are described in detail below with reference to the accompanying drawings.

[0034] Figure 1 The logical architecture diagram of the web-based interactive data visualization creation method of the present invention is combined with Figure 1 The web-based interactive data visualization creation method of the present invention is described in detail, comprising the following steps:

[0035] 1. The user enters search information in the terminal. The distributed web crawler tool obtains the corresponding web page data set and web page identifier based on the search information entered by the user, and puts the web page identifier into the first task queue.

[0036] Among them, the web page data set is a massive amount of web page data to be crawled based on the search information typed by the user, which corresponds to multiple web pages. The role of the distributed web crawler tool is to crawl the multi-level web pages corresponding to these data. The web page identifier can be understood as the web page URL. The web page identifier and the web page URL below have the same technical connotation.

[0037] The massive amount of web page data on the Internet is linked to different URLs in a certain way, and these URLs are distributed in a tree structure. The URL at the top of the tree structure is the first-level web page URL, which is linked to multiple second-level web page URLs. Each second-level web page URL is linked to multiple third-level web page URLs, and each third-level web page URL is linked to multiple fourth-level web page URLs, and so on. The URL at the bottom of the tree structure has no web page URLs linked to it.

[0038] The distributed web crawler tool obtains the corresponding web page identifier from a massive web page data set in the following manner: first, obtains the first-level web page URL based on the web page data set; obtains multiple second-level web page URLs under the first-level web page URL, and sequentially places them in the first task queue; for each second-level web page URL, obtains multiple third-level web page URLs under it, and sequentially places them in the first task queue; for each third-level web page URL, obtains multiple fourth-level web page URLs under it, and sequentially places them in the first task queue; and so on, until the web page URLs of all sub-web pages under the first-level web page URL are placed in the first task queue.

[0039] It is understandable that due to the overlapping hierarchical relationships between different web page URLs, duplicate URLs need to be eliminated in the process of obtaining web page URLs; however, since the present invention crawls hierarchical web page URLs through a distributed web crawler tool, there is no duplication between URLs at different levels. Therefore, when identifying and eliminating duplicate URLs, it is only necessary to identify and eliminate them between web page URLs at the same level that are crawled.

[0040] Thus, the present invention, by introducing a distributed web crawler tool, avoids acquiring duplicate web page data and improves the efficiency of subsequent web page data downloads. Furthermore, during each crawling process, it only needs to determine whether there are duplicate URLs within the current level, which improves the efficiency of duplicate URL identification and avoids the drawback of acquiring a large number of duplicate URLs, which would otherwise complicate and duplicate the subsequent web page data download process.

[0041] 2. Build multiple groups of servers and match the web page identifiers (i.e., web page URLs) in the first task queue with the multiple groups of servers.

[0042] like Figure 1 As shown, in the multiple server groups constructed by the present invention, each server group includes a proxy server and a download server. The proxy server has a powerful cache function, wherein the cache has web page data within a certain period of time, and the download server is used to communicate with a specific web page server (corresponding to Figure 1 The web server 1, the web server 2, ..., the web server n) are connected to download the required web page data from the web server.

[0043] When matching the web page URLs in the first task queue with multiple groups of servers, a web page URL is taken out from the first task queue in sequence as the current web page URL, and the proxy servers in the multiple groups of servers are traversed: if a proxy server has web page data corresponding to the current web page URL cached, the current web page URL is matched to the server to which the proxy server belongs; if all proxy servers do not have web page data corresponding to the current web page URL cached, the current web page URL is matched to the corresponding server according to the matching order of the multiple groups of servers.

[0044] For example, assuming that there are five groups of servers, namely the first server, the second server, the third server, the fourth server and the fifth server, under normal circumstances, the current web page URL taken out from the first task queue will be matched with the corresponding server in the order of the first server, the second server, the third server, the fourth server and the fifth server. Only when the web page data required by the current web page URL is cached in the proxy server of one of the servers, this relationship will be broken, and the current web page URL will be matched with the server to which the proxy server that has cached the required web page data belongs; for example, the first web page URL is taken out from the first task queue as the current web page URL, and the second server If the proxy server of the first to fifth servers has cached the web page data corresponding to the current web page URL, the current web page URL is matched to the second server; then, the second web page URL is taken out from the first task queue as the current web page URL, and none of the proxy servers among the first to fifth servers has cached the web page data corresponding to the current web page URL, then the current web page URL is matched to the first server according to the matching order of the servers; then, the third web page URL is taken out from the first task queue as the current web page URL, and none of the proxy servers among the first to fifth servers has cached the web page data corresponding to the current web page URL, then the current web page URL is matched to the second server according to the matching order of the servers, and so on.

[0045] Thus, in the present invention, by constructing multiple groups of servers to concurrently download web page data, the download efficiency of web page data is improved, the efficiency of system feedback of web page data and rendering and generating web pages is improved, and the decoupling of the system is enhanced.

[0046] Moreover, when the distributed web crawler tool obtains the relevant web page URL, it puts the web page URL into the task queue in turn by building a task queue and a server, and matches it with the server. The message communication mechanism between the distributed web crawler tool and the server cluster is built through the task queue, avoiding the problems of heavy coupling, high resource consumption and cumbersome information configuration brought by address port communication.

[0047] 3. Each server downloads the web page data based on the matched web page identifier (ie, web page URL), and sequentially places the web page identifier (ie, web page URL) and the downloaded web page data into the second task queue.

[0048] When the server downloads web page data, if the proxy server has web page data corresponding to the web page URL cached in it, the web page data is directly obtained from the cache, and the web page URL and the web page data are sequentially placed in the second task queue; if the proxy server does not have web page data corresponding to the web page URL cached in it, the proxy server sends the web page URL to the download server, and the download server downloads the web page data from the corresponding web page server based on the web page URL, and feeds it back to the proxy server. The proxy server caches the obtained web page data, and then places the web page URL and the web page data in sequence in the second task queue.

[0049] In this way, the powerful cache function of the proxy server is utilized to avoid resource consumption caused by repeated downloading of cached web page data, further improving the download efficiency of web page data.

[0050] Similarly, the server puts the downloaded web page data into the task queue in sequence, and builds a message communication mechanism between the terminal and the server cluster through the task queue, avoiding the problems of heavy coupling, high resource consumption and cumbersome information configuration brought by address port communication.

[0051] 4. Identify the web page identifiers (i.e., web page URLs) and web page data in the second task queue, and discard web page data related to junk web pages.

[0052] Specifically, in a preferred embodiment of the present invention, web page data related to spam web pages is identified through the following steps:

[0053] (1) Representing the features based on the web page URL as a first URL matrix, and representing the features based on the web page data as a first data matrix;

[0054] (2) Projecting the first data matrix through the optimal first data projection matrix to obtain a second data matrix, and projecting the first address matrix through the optimal first address projection matrix to obtain a second address matrix; wherein, the method for obtaining the optimal first data projection matrix and the optimal first address projection matrix is: obtaining multiple sets of data projection matrices and address projection matrices by using multiple typical correlation analysis methods on the first data matrix and the first address matrix, and selecting the data projection matrix and address projection matrix with the largest correlation as the optimal first data projection matrix and the optimal first address projection matrix;

[0055] (3) Merging the second data matrix and the second URL matrix to obtain a first input matrix;

[0056] (4) Inputting the first input matrix into the trained neural network model, and judging whether the web page data involves spam web pages based on the output result of the neural network model.

[0057] That is, when the present invention identifies junk web pages, the main object identified is not the final generated web page, but the URL and web page data corresponding to the web page. In other words, for junk web pages, the present invention directly discards them from the second task queue before rendering. Compared with the existing mechanism for identifying junk web pages based on web page content, the web page rendering step is directly omitted, and the hardware resource consumption of web page rendering is omitted, which greatly saves resource consumption.

[0058] In the present invention, the training method of the above-mentioned neural network model includes:

[0059] (1) preparing a training set, which includes multiple sets of web page URLs and web page data corresponding to the web page URLs, and each set of web page URLs and web page data is marked, and the marking result is used to indicate whether the web page data involves spam web pages;

[0060] (2) For each set of web page URLs and web page data in the training set, the features based on the web page URLs are represented as a third URL matrix, and the features based on the web page data are represented as a third data matrix;

[0061] (3) Projecting the third data matrix by the optimal second data projection matrix to obtain a fourth data matrix, and projecting the third data matrix by the optimal second data projection matrix to obtain a fourth data matrix;

[0062] (4) Merging the fourth data matrix and the fourth URL matrix to obtain a second input matrix;

[0063] (5) For each set of web page URLs and web page data in the training set, the labeling results obtained based on step (1) and the second input matrix obtained in step (4) are input into the neural network model to train the neural network model.

[0064] In the present invention, the various canonical correlation analysis methods may include kernel canonical correlation analysis, local preserving canonical correlation analysis, and deep canonical correlation analysis, etc., and the present invention is not limited thereto.

[0065] 5. Render the web page data in the second task queue in sequence, generate the final web page and display it to the user on the terminal.

[0066] In the present invention, in order to optimize and improve the accuracy of neural network model recognition, a visual annotation menu can be constructed on the generated web page for users to annotate the web page on the visual annotation menu, and the neural network model can be adjusted based on the annotation results.

[0067] Specifically, a visual annotation menu can be constructed by adding cascading style sheets (CSS) and javascript scripts to the source code of a web page. After construction, when a user browses a web page, a marking option can be displayed on the web page by clicking the mouse. The user can choose to mark whether the currently browsed web page is a spam web page (such as an advertising web page). For web pages marked as spam by users, they are fed back to the neural network model as training sets, and the neural network model conducts further learning and fine-tunes parameters to further improve the accuracy of identifying subsequent spam web pages.

[0068] Furthermore, the present invention can also render the web page data involving spam web pages identified in step 4 to generate a final web page; accordingly, the display interface of the terminal is divided into two areas: a first display area and a second display area, the first display area displays the web page finally generated in step 5, and the second display area displays the web page generated after rendering the web page data involving spam web pages identified in step 4; it is obvious that the proportion of the first display area is significantly larger than that of the second display area; the setting of the second display area is to avoid the neural network model from mistakenly identifying certain normal web pages as spam web pages. In this way, a second display area is provided, and when the user finds that there are normal web pages therein, it can also be marked as a normal web page. For the web pages marked as normal by the user, they are also fed back to the neural network as a training set, and the neural network model further learns and fine-tunes the parameters to further improve the recognition accuracy of subsequent spam web pages.

[0069] Therefore, the present invention provides an interactive mechanism for users through the construction of a visual annotation menu, which is convenient for accepting user feedback on the finally generated web page and adjusting the neural network model, thereby improving the recognition accuracy of the neural network; and, through the setting of the first display area and the second display area, it is not only convenient for users to mark junk web pages missed in the neural network model recognition process, but also convenient for users to mark junk web pages that are mistakenly marked in the neural network model recognition process.

[0070] In the present invention, in order to further save hardware resources, different rendering effects can be given when rendering web pages based on different configuration information of the terminal. For example, the type of terminal, the network environment in which the terminal is located, and the hardware resource information of the terminal are obtained to obtain a system configuration parameter, and the rendering type is determined according to the system configuration parameter.

[0071] The present invention can set a first rendering type, a second rendering type and a third rendering type: the first rendering type has the highest requirement for image resolution, provides the clearest window view, and has the largest traffic consumption and hardware resource consumption. This type is suitable for desktops and tablets in a wifi environment with sufficient hardware resources; the second rendering type has moderate requirements for image resolution, provides a relatively clear window view, and has average traffic consumption and hardware resource consumption. This type is suitable, for example, for tablets and laptops in a 5G network with relatively sufficient hardware resources; the third rendering type has the lowest requirement for image resolution, provides the fuzziest window view, and has the least traffic consumption and hardware resource consumption. This type is suitable, for example, for mobile terminals in a 2G network.

[0072] The present invention can determine the system configuration parameters of the terminal by, for example, the following formula:

[0073] System configuration parameter = α X+β Y+γ Z;

[0074] Among them, α, β, and γ represent the weights of network environment, terminal type, and hardware resources in evaluating system configuration parameters; the optimal values ​​are 0.2, 0.4, and 0.4 respectively;

[0075] X, Y, and Z represent the network environment parameter value, terminal type parameter value, and hardware resource parameter value, respectively;

[0076] The network environment parameter value can be determined according to the network environment of the terminal. For example, the values ​​of Wi-Fi network, 5G network and 2G network can be assigned as 0.5, 0.4 and 0.1 respectively;

[0077] Similarly, the terminal type parameter value can also be determined according to the type of terminal. For example, desktop computers, tablet computers, laptop computers, and mobile phones can be assigned values ​​of 0.9, 0.8, 0.8, and 0.5, respectively;

[0078] The hardware resource parameter value can be determined based on the CPU's main frequency, the number of GPU hardware cores, and the RAM capacity. In a preferred embodiment, based on actual needs, the CPU's main frequency, the number of GPU hardware cores, and the RAM capacity can be divided into three value intervals: a high interval, a middle interval, and a low interval. The corresponding hardware resource volume decreases from the high interval to the low interval. The high interval, middle interval, and low interval of the CPU's main frequency can be assigned parameter values ​​of 0.8, 0.6, and 0.4, respectively. The high interval segment, middle interval segment, and low interval segment of the PU hardware core number can be assigned parameter values ​​of 1.0, 0.8, and 0.6, respectively; the high interval segment, middle interval segment, and low interval segment of the RAM capacity can be assigned parameter values ​​of 0.6, 0.4, and 0.2, respectively. Finally, determine in which interval segment the actual values ​​of the terminal CPU's main frequency, the GPU's hardware core number, and the RAM capacity fall, obtain the parameter values ​​of the CPU's main frequency, the GPU's hardware core number, and the RAM capacity, and take the average to obtain the hardware resource parameter value.

[0079] Since the degree of influence of the CPU, GPU and RAM on web page rendering is taken into consideration when allocating parameter values ​​to different intervals of the CPU's main frequency, the number of GPU hardware cores and the RAM capacity, the parameter values ​​allocated to the CPU's main frequency, the number of GPU hardware cores and the RAM capacity are different within the same interval. Therefore, the parameter values ​​of the CPU's main frequency, the number of GPU hardware cores and the RAM capacity finally obtained are equivalent to having taken into account the degree of influence of the CPU, GPU and RAM on web page rendering. Therefore, when determining the final parameter values ​​of the hardware resources, the parameter values ​​of the CPU's main frequency, the number of GPU hardware cores and the RAM capacity can be directly averaged.

[0080] Finally, when determining the correspondence between system configuration parameters and rendering types, the system configuration parameters can also be divided into three different intervals: a high interval, a middle interval, and a low interval. The three intervals correspond to the first rendering type, the second rendering type, and the third rendering type, respectively.

[0081] Through the above complete network data acquisition and feedback mechanism, the present invention can achieve the following technical effects:

[0082] 1. Through distributed web crawler tools, web page URLs are crawled from massive web page data according to the level of the URLs, which improves the systematicness of obtaining web page URLs. In each crawling process, it is only necessary to determine whether there are duplicate URLs between the URLs at the current level, which improves the efficiency of duplicate URL identification and avoids the defect of obtaining a large number of duplicate URLs, which leads to the complicated and repetitive process of subsequent web page data downloading.

[0083] 2. By building multiple groups of servers to download web page data concurrently, the efficiency of web page data downloading is improved, the efficiency of system feedback of web page data and rendering and generating web pages is improved, and the system decoupling is enhanced.

[0084] 3. By building two sets of task queues, the message communication mechanism between the server and the distributed web crawler tool and the terminal is completed, avoiding the problems of heavy coupling, high resource consumption and cumbersome information configuration caused by address port communication.

[0085] 4. Through the powerful caching function of the proxy server, when matching web page URLs and servers, the web page URLs are matched to the server where the proxy server that caches the corresponding web page data is located, saving resource consumption for downloading web page data and further improving the download efficiency of web page data.

[0086] 5. By building a neural network model, web page data can be directly identified and junk web pages can be filtered out before rendering and generating web pages. This not only greatly improves the user experience, but also greatly saves the rendering cost of junk web pages, further saves hardware resource consumption, and improves the system's work efficiency.

[0087] 6. By building a visual annotation menu, a feedback adjustment mechanism is provided to facilitate the acceptance of user feedback results, so as to fine-tune the neural network model based on the feedback results, form a closed-loop mechanism, and continuously improve the accuracy of spam web page identification.

[0088] 7. The rendering type is determined based on the terminal's network environment, terminal type, and hardware resource volume, providing a customized technical solution for providing web pages, ensuring the match between the web page provided and the terminal configuration information to the greatest extent possible, and providing users with the best user experience within the capabilities of the terminal.

[0089] In summary, the web-based interactive data visualization creation method provided by the present invention is a systematic mechanism for web URL crawling, web data downloading, junk web page identification, web page generation and feedback. Through various technical means, it improves the download efficiency of web data from multiple dimensions and reduces resource consumption in the web page rendering process. At the same time, it can combine with the user feedback mechanism to continuously optimize the model screening accuracy, thereby maximizing the user experience.

[0090] Although the present invention has been described using the above preferred embodiments, they are not intended to limit the scope of protection of the present invention. Any person skilled in the art may make various changes and modifications to the above embodiments without departing from the spirit and scope of the present invention. These changes and modifications are still within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the claims.

Claims

1. A web-based interactive data visualization creation method, characterized in that: The steps include: Step S1: Obtain a web page dataset based on user search information; Step S2: Based on the web page dataset, obtain a first-level web page identifier; Step S3: traverse the webpage IDs of all sub-webpages under the first-level webpage ID, and put the webpage IDs of all sub-webpages into the first task queue in sequence; Step S4: construct multiple groups of servers, and match the web page identifiers in the first task queue with the multiple groups of servers; Step S5: Each server downloads the web page data based on the matched web page identifier, and sequentially places the web page identifier and the downloaded web page data into the second task queue; Step S6: Identify the web page identifiers and web page data in the second task queue, and discard web page data related to junk web pages; Step S7: Rendering the web page data in the second task queue in sequence to generate and display the final web page; When rendering a web page, the rendering type is determined according to the system configuration parameters of the terminal, wherein the rendering type is the first rendering type, the second rendering type, or the third rendering type. The system configuration parameters are obtained by the following formula: System configuration parameter = α X+β Y+γ Z; Among them, α, β, and γ represent the weights of network environment, terminal type, and hardware resources in evaluating system configuration parameters respectively; X, Y, and Z represent the network environment parameter value, terminal type parameter value, and hardware resource parameter value, respectively; The network environment parameter value is determined according to the network environment in which the terminal is located; The terminal type parameter value is determined according to the type of terminal; The hardware resource parameter value is determined based on the terminal's CPU main frequency, GPU hardware core number, and RAM capacity. When determining the hardware resource parameter value, the CPU main frequency, GPU hardware core number, and RAM capacity are divided into high, middle, and low intervals, and the corresponding hardware resource volume decreases from the high interval to the low interval. Based on the actual values ​​of the terminal's CPU main frequency, GPU hardware core number, and RAM capacity, the parameter values ​​of the CPU main frequency, GPU hardware core number, and RAM capacity are obtained, and the average is taken to obtain the hardware resource parameter value. The CPU's main frequency, the number of GPU cores, and the RAM capacity have different corresponding parameter values ​​in their respective high-order intervals; The CPU main frequency, GPU hardware core number and RAM capacity have different corresponding parameter values ​​in their respective median intervals; The CPU's main frequency, the number of GPU hardware cores, and the RAM capacity have different corresponding parameter values ​​in their respective low-order intervals.

2. The web-based interactive data visualization creation method according to claim 1, wherein: In step S4, each of the multiple server groups constructed includes a proxy server and a download server. The proxy server caches certain web page data, and the download server is used to connect to a specific web page server to download the required web page data from the web page server. When matching the web page identifiers in the first task queue with multiple groups of servers, a web page identifier is taken out from the first task queue in sequence as the current web page identifier, and the proxy servers in the multiple groups of servers are traversed: if a proxy server has web page data corresponding to the current web page identifier cached, the current web page identifier is matched to the server to which the proxy server belongs; if all proxy servers do not have web page data corresponding to the current web page identifier cached, the current web page identifier is matched to the corresponding server according to the matching order of the multiple groups of servers.

3. The web-based interactive data visualization creation method according to claim 2, wherein: In step S5, when the server downloads the web page data based on the matched web page identifier: If the proxy server has web page data corresponding to the web page identifier cached in the proxy server, the web page identifier and the web page data are directly placed in the second task queue in sequence; If the proxy server does not cache the web page data corresponding to the web page identifier, the proxy server sends the web page identifier to the download server, and the download server downloads the web page data from the corresponding web page server based on the web page identifier.

4. The web-based interactive data visualization creation method according to claim 1, wherein: In step S6, web page data related to junk web pages is identified through the following steps: Step S61: representing features based on web page identifiers as a first URL matrix, and representing features based on web page data as a first data matrix; Step S62: Projecting the first data matrix using the optimal first data projection matrix to obtain a second data matrix, and projecting the first data matrix using the optimal first data projection matrix to obtain a second data matrix; Step S63: merging the second data matrix and the second URL matrix to obtain a first input matrix; Step S64: inputting the first input matrix into the trained neural network model, and determining whether the web page data involves junk web pages based on the output result of the neural network model.

5. The web-based interactive data visualization creation method according to claim 4, wherein: In step S62, the method for obtaining the optimal first data projection matrix and the optimal first URL projection matrix is: the first data matrix and the first URL matrix are subjected to multiple typical correlation analysis methods to obtain multiple sets of data projection matrices and URL projection matrices, and the data projection matrix and URL projection matrix with the greatest correlation are selected as the optimal first data projection matrix and the optimal first URL projection matrix.

6. The web-based interactive data visualization creation method according to claim 4, wherein: In step S64, the training method of the neural network model includes: Step S641: preparing a training set, the training set including multiple sets of web page identifiers and web page data corresponding to the web page identifiers, and each set of web page identifiers and web page data is marked, and the marking result is used to indicate whether the web page data involves spam; Step S642: for each set of web page identifiers and web page data in the training set, representing features based on the web page identifiers as a third URL matrix, and representing features based on the web page data as a third data matrix; Step S643: Projecting the third data matrix using the optimal second data projection matrix to obtain a fourth data matrix, and projecting the third data matrix using the optimal second data projection matrix to obtain a fourth data matrix; Step S644: merging the fourth data matrix and the fourth URL matrix to obtain a second input matrix; Step S645: For each set of web page identifiers and web page data in the training set, the labeling result obtained based on step S641 and the second input matrix obtained in step S644 are input into the neural network model to train the neural network model.

7. The web-based interactive data visualization creation method according to claim 4, wherein: The method further includes step S8: constructing a visual annotation menu in the generated final web page, allowing the user to annotate the web page on the visual annotation menu, and adjusting the neural network model based on the annotation results.

8. The web-based interactive data visualization creation method according to claim 7, wherein: In step S8, a visual annotation menu is constructed by adding a cascading style sheet and a javascript script to the source code of the web page.

Images